Focus adjusting device and focus adjusting program with distribution detection of focalized and unfocused state

ABSTRACT

A focus adjusting device which includes an edge detection unit that detects edges of a subject image for each color component forming an image including the subject image; a distribution detection unit that detects distributions of a focalized state and an unfocused state of the image based on the edges detected by the edge detection unit; and a control unit that moves a lens based on the distributions detected by the distribution detection unit, wherein the subject image is incident from an optical system having the lens, the control unit moves the lens, and thus the subject image is focused on.

The present application is a continuation application of U.S. patentSer. No. 13/578,927, filed on Aug. 14, 2012, which is a national stageof PCT/JP2011/053156, filed Feb. 15, 2011. Priority is claimed toJapanese Patent Application No. 2010-030481, filed Feb. 15, 2010. Thecontent of each of the above references is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present invention relates to a focus adjusting device which adjustsfocus by applying chromatic aberration, and a focus adjusting program.

BACKGROUND ART

A video camera which applies chromatic aberration to focus a subjectimage obtained through a lens is known (refer to Patent Document 1).

However, the video camera disclosed in Patent Document1 can rapidlyfocus on a subject image which has previously been focused on, butcannot rapidly focus on a subject image which has not previously beenfocused on. That is to say, there is a problem in that the video cameradisclosed in Patent Document1 cannot rapidly focus on a subject image.

CITATION LIST Patent Document

[Patent Document 1] Japanese Unexamined Patent Application, FirstPublication No. H5-45574

SUMMARY OF INVENTION Technical Problem

An object of the present invention is to provide a focus adjustingdevice and a focus adjusting program enabling a subject image to berapidly focused on by applying chromatic aberration.

Solution to Problem

A focus adjusting device according to an aspect of the present inventionincludes an edge detection unit that detects edges of a subject imagefor each color component forming an image including the subject image; adistribution detection unit that detects distributions of a focalizedstate and an unfocused state of the image based on the edges detected bythe edge detection unit; and a control unit that moves a lens based onthe distributions detected by the distribution detection unit. Thesubject image is incident from an optical system having the lens. Inaddition, the control unit moves the lens and thus the subject image isfocused on.

The focus adjusting device may be configured as follows: thedistribution detection unit detects the distributions based on agradient of a color component amount of the edges detected by the edgedetection unit.

The focus adjusting device may be configured as follows: thedistribution detection unit detects a direction index indicatingfocalization on a close point side or a distant point side with respectto a subject, and a defocus amount, based on a ratio of or a differencebetween color component amounts of the edges detected by the edgedetection unit.

The focus adjusting device may be configured as follows: thedistribution detection unit detects the defocus amount based on adistance between peaks of the ratios of or the difference between thecolor component amounts of the edges detected by the edge detectionunit.

The focus adjusting device may be configured as follows: thedistribution detection unit detects a direction index indicatingfocalization on a close point side or a distant point side with respectto a subject, and a defocus amount, based on a line spread functioncorresponding to the edges detected by the edge detection unit.

The focus adjusting device may be configured as follows: thedistribution detection unit detects the defocus amount, based on astandard deviation or a full width at half maximum of the line spreadfunction corresponding to the edges detected by the edge detection unit.

The focus adjusting device may be configured as follows: thedistribution detection unit selects a high-ranking edge in descendingorder of a color component amount from the edges detected by the edgedetection unit, and detects the distributions of a focalized state andan unfocused state of the image based on the selected edge.

The focus adjusting device may be configured as follows: thedistribution detection unit selects a high-ranking edge in descendingorder of a contrast of the color component from the edges detected bythe edge detection unit, and detects the distributions of a focalizedstate and an unfocused state of the image based on the selected edge.

The focus adjusting device may be configured as follows: when edges ofwhich the contrasts of the color component are the same and edges ofwhich a signal to noise ratios are different are mixed, the distributiondetection unit selects a high-ranking edge in descending order of thesignal to noise ratio of the color component, and detects thedistributions of a focalized state and an unfocused state of the imagebased on the selected edge.

The focus adjusting device may be configured as follows: when edges ofwhich the signal to noise ratios are different are mixed, thedistribution detection unit selects at least one edge which has arelatively low signal to noise ratio and a relatively high contrast andan edge which has a relatively high signal to noise ratio and arelatively low contrast, and detects the distributions of a focalizedstate and an unfocused state of the image based on the selected edge.

The focus adjusting device may be configured as follows: thedistribution detection unit selects edges which include two or moreprimary colors and have a color component varying with the same phasefrom the edges detected by the edge detection unit, and detects thedistributions of a focalized state and an unfocused state of the imagebased on the selected edges.

The focus adjusting device may be configured as follows: thedistribution detection unit selects an edge including a green componentwhen the edges include two primary colors.

The focus adjusting device may be configured as follows: thedistribution detection unit selects an edge which has a flat colorcomponent at a predefined width or more from the edges detected by theedge detection unit, and detects the distributions of a focalized stateand an unfocused state of the image based on the selected edge.

The focus adjusting device may be configured as follows: thedistribution detection unit selects an edge having a length or moredefined according to a signal to noise ratio of the color component fromedges for each color component detected by the edge detection unit, anddetects the distributions of a focalized state and an unfocused state ofthe image based on the selected edge.

A focus adjusting device related to an aspect of the present inventionincludes an edge detection unit that detects edges of a subject imagefor each color component forming an image including the subject image; adistribution detection unit that calculates a line spread function ofthe edges detected by the edge detection unit; and a control unit thatmoves a lens based on the line spread function. The subject image isincident from an optical system having the lens. In addition, thecontrol unit moving the lens and thus the subject image is focused on.

A computer related to an aspect of the present invention is a computerexecuting a focus adjusting program. The focus adjusting programincludes a step of detecting edges of a subject image for each colorcomponent forming an image including the subject image which is incidentfrom an optical system having a lens for performing focus adjustment; astep of detecting distributions of a focalized state and an unfocusedstate of the image based on the edges; and a step of moving the lens soas to focus on the subject image based on the distributions.

A computer related to an aspect of the present invention is a computerexecuting a focus adjusting program. The focus adjusting programincludes a step of detecting edges of a subject image for each colorcomponent forming an image including the subject image which is incidentfrom an optical system having a lens for performing focus adjustment; astep of calculating a line spread function of the edges; and a step ofmoving the lens so as to focus on the subject image based on the linespread function.

Advantageous Effects of Invention

The focus adjusting device and the focus adjusting program related tothe present invention achieve an effect that a subject image can berapidly focused on.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating configurations of an imagingapparatus 100 having a lens barrel 111 and a focus adjusting device 191and a storage medium 200. FIG. 2 is a diagram illustrating arelationship between a subject at a position separated from an AF lens112 by a subject distance, the AF lens 112, an imaging surface of animaging element 119, and the circle of confusion.

FIG. 3 is a diagram illustrating a positional relationship on theoptical axis among a focal point of red light, a focal point of greenlight, and a focal point of blue light, incident to the AF lens 112.

FIG. 4 is a diagram illustrating a relationship between a wavelength oflight incident to the AF lens 112 and a position of the imaging surfaceon the optical axis.

FIG. 5A is a diagram illustrating a relationship between the AF lens112, the imaging surface of the imaging element 119, and the circle ofconfusion in a front focus state.

FIG. 5B is a diagram illustrating the circle of confusion in a frontfocus state, formed on the imaging surface.

FIG. 5C is a diagram illustrating a relationship between a position inthe radial direction toward the outer circumference from the center ofthe circle of confusion in a front focus state and a color componentamount.

FIG. 6A is a diagram illustrating a relationship between the AF lens112, the imaging surface of the imaging element 119, and the circle ofconfusion in a back focus state.

FIG. 6B is a diagram illustrating the circle of confusion in a backfocus state, formed on the imaging surface.

FIG. 7 is a diagram illustrating an example of the captured image.

FIG. 8A is a diagram illustrating a color component amount in thevicinity of an edge position 13 and is a grayscale cross-sectional viewin a focalized state with a G channel.

FIG. 8B is a diagram illustrating a color component amount in thevicinity of the edge position 13 and is a grayscale cross-sectional viewin a focalized state with an R channel.

FIG. 8C is a diagram illustrating a color component amount in thevicinity of the edge position 13 and is a grayscale cross-sectional viewin a focalized state with a B channel.

FIG. 9A is a diagram illustrating a color component amount in thevicinity of the edge position 13, and shows a relationship between anedge of the R channel in a front focus state, an edge of the R channelin a back focus state, and an edge of the G channel in a focalizedstate.

FIG. 9B is a diagram illustrating a color component amount in thevicinity of the edge position 13, and shows a relationship between anedge of the B channel in a front focus state, an edge of the B channelin a back focus state, and an edge of the G channel in a focalizedstate.

FIG. 10A is a diagram illustrating a “color component amount difference”in the vicinity of the edge position 13, and shows a relationshipbetween a position at an axis D and a difference between a colorcomponent amount of the R channel and a color component amount of the Gchannel.

FIG. 10B is a diagram illustrating a “color component amount difference”in the vicinity of the edge position 13, and shows a waveform in a caseof being close to a focalized state as compared with FIG. 10A.

FIG. 11A is a diagram illustrating an LSF in a focalized state in the Gchannel.

FIG. 11B is a diagram illustrating an LSF in an unfocused (small blur)state in the G channel.

FIG. 11C is a diagram illustrating an LSF in an unfocused (large blur)state in the G channel.

FIG. 12A is a diagram illustrating an LSF in a focalized state for eachcolor channel (R, G, and B).

FIG. 12B is a diagram illustrating an LSF in a front focus state foreach color channel (R, G, and B).

FIG. 12C is a diagram illustrating an LSF in a back focus state for eachcolor channel (R, G, and B).

FIG. 13 is a diagram illustrating a relationship between a profile(data) of an LSF for each color channel (R, G, and B) and a blur degree.

FIG. 14 is a diagram illustrating a relationship between a functionvalue having a difference between standard variations of the LSF and asubject distance.

FIG. 15A shows an example of a part (partial image) of the imagecaptured by the imaging unit.

FIG. 15B shows an edge image of the R channel extracted from thecaptured image.

FIG. 15C shows an edge image of the G channel extracted from thecaptured image.

FIG. 15D is an image indicating a calculated result of logical product(AND) of the edge image of the R channel and the edge image of the Gchannel.

FIG. 16 is a flowchart illustrating an operation of an edge detectionunit 192.

FIG. 17 is an example of the diagram illustrating a color componentamount in the vicinity of the edge position 13 and is a diagramillustrating a calculation example of using a Laplacian filter.

FIG. 18 is a diagram illustrating an example of the depth map.

FIG. 19 is a flowchart illustrating a procedure of the focus drive ofthe focus adjusting device 191 in a tracking operation of tracking asubject.

FIG. 20 is a flowchart illustrating an analysis procedure of the edgecharacteristics.

FIG. 21 is a diagram illustrating a scanning region when a front focusstate is determined and a scanning region when a back focus state isdetermined.

FIG. 22A is a diagram illustrating a movement of a lens position inhill-climbing contrast scanning during AF for photographing, and showsan example of the relationship between a position of the AF lens 112 anda contrast value in the contrast scanning.

FIG. 22B shows a movement of a position of the AF lens 112 in normalcontrast scanning (refer to the middle part of FIG. 21).

FIG. 22C shows a movement of a position of the AF lens 112 in contrastscanning (refer to the lower part of FIG. 21) based on determination ofa front focus state and a back focus state depending on an evaluationvalue.

FIG. 23 is a flowchart illustrating a procedure of the focus drive basedon the LSF.

FIG. 24 is a diagram illustrating an example of the “defocus-drivingpulse table”.

FIG. 25 is a diagram illustrating a contrast scanning region when frontfocus and a degree thereof are determined and a contrast scanning regionwhen back focus and a degree thereof are determined.

FIG. 26 is a flowchart illustrating a procedure of the focus drive ofthe focus adjusting device 191 in a tracking operation of tracking asubject.

FIG. 27 is a diagram illustrating an example of the focus drive in thecontrast scanning based on the “defocus-driving pulse table”.

FIG. 28 is an enlarged view of an example of the focus drive in thecontrast scanning based on the “defocus-driving pulse table”.

FIG. 29 is a flowchart illustrating an operation of the focus adjustingdevice 191 having a moving image capturing mode.

FIG. 30A is a diagram illustrating a histogram of difference data in afocalized state.

FIG. 30B is a diagram illustrating a histogram of difference data in anunfocused state.

FIG. 31 is a flowchart illustrating an operation of the focus adjustingdevice 191 of determining spot light.

FIG. 32 is a diagram illustrating an operation of determining conversioninto a macro imaging mode.

FIG. 33A is a diagram illustrating a priority of an edge selected in acase where edges of which signal to noise ratios of color components aredifferent are mixed, and shows a case of an edge having a relativelyhigh signal to noise ratio and a relatively low contrast.

FIG. 33B is a diagram illustrating a priority of an edge selected in acase where edges of which signal to noise ratios of color components aredifferent are mixed, and shows a case of an edge having a relatively lowsignal to noise ratio and a relatively high contrast.

FIG. 34A is a diagram illustrating a priority of an edge selected whenincluding two primary colors, and shows a case of an edge formed by theR channel and the G channel which vary with the same phase.

FIG. 34B is a diagram illustrating a priority of an edge selected whenincluding two primary colors. The edge having a priority shown in FIG.34B is formed by the R channel and the G channel varying with the samephase, wherein the contrast of the R channel is lower than the contrastof the R channel shown in FIG. 34A.

FIG. 34C is a diagram illustrating a priority of an edge selected whenincluding two primary colors. FIG. 34C shows a case of an edge formed bythe R channel and the B channel varying with the same phase.

FIG. 35A shows a variation in a color component amount corresponding toa position crossing a single edge having a flat color component.

FIG. 35B shows a variation in a color component amount corresponding topositions crossing a plurality of edges.

FIG. 36 is a flowchart illustrating a procedure of selecting ahigh-ranking edge according to the priority.

DESCRIPTION OF EMBODIMENTS First Embodiment

The first embodiment of the present invention will be described indetail with respect to the drawings. FIG. 1 is a block diagramillustrating configurations of an imaging apparatus 100 including a lensbarrel 111 and a focus adjusting device 191 and a storage medium 200.The imaging apparatus 100 captures a subject image which is incidentfrom the lens barrel 111 and stores the obtained image in the storagemedium 200 as a still image or a moving image.

First, a configuration of the lens barrel 111 will be described. Thelens barrel 111 includes a focus adjusting lens (hereinafter, referredto as an “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AFencoder 117, and a barrel control unit 118. In addition, the lens barrel111 may be connected to the imaging apparatus 100 so as to be attachableand detachable, or may be integrally formed with the imaging apparatus100.

The AF lens 112 is driven by the lens driving unit 116. The AF lens 112guides a subject image to a light receiving surface (photoelectricconversion surface) of an imaging element 119 of an imaging unit 110described later.

The AF encoder 117 detects a movement of the AF lens 112, and outputs asignal corresponding to a movement amount of the AF lens 112 to thebarrel control unit 118. Here, the signal corresponding to a movementamount of the AF lens 112 may be, for example, a sine (sin) wave signalof which a phase varies according to a movement amount of the AF lens112.

The barrel control unit 118 controls the lens driving unit 116 inresponse to a driving control signal input from the focus adjustingdevice 191, described later, of the imaging apparatus 100. Here, thedriving control signal is a control signal for driving the AF lens 112in the optical axis direction. The barrel control unit 118 changes, forexample, the number of steps of a pulse voltage which is output to thelens driving unit 116, in response to the driving control signal.

In addition, the barrel control unit 118 outputs a position (focusposition) of the AF lens 112 in the lens barrel 111 to the focusadjusting device 191 described later, based on the signal correspondingto a movement amount of the AF lens 112. Here, the barrel control unit118 may calculate a movement amount (position) of the AF lens 112 in thelens barrel 111, for example, by adding up the signal according to amovement amount of the AF lens 112 in the movement direction.

The lens driving unit 116 drives the AF lens 112 under the control ofthe barrel control unit 118. In addition, the lens driving unit 116moves the AF lens 112 in the optical axis direction inside the lensbarrel 111.

Next, a configuration of the imaging apparatus 100 will be described.The imaging apparatus 100 includes an imaging unit 110, an imageprocessing unit 140, a display unit 150, a buffer memory unit 130, anoperation unit 180, a storage unit 160, a CPU 190, a communication unit170, and the focus adjusting device 191.

The imaging unit 110 includes an imaging element 119 and an A/D(Analog/Digital) conversion unit 120. The imaging unit 110 is controlledby the CPU 190 depending on set imaging conditions (for example, adiaphragm value, an exposure value, and the like).

The imaging element 119 has a photoelectric conversion surface. Theimaging element 119 converts an optical image formed on thephotoelectric conversion surface by the lens barrel 111 (optical system)into an electric signal which is output to the A/D conversion unit 120.The imaging element 119 may be made of, for example, CMOS (ComplementaryMetal Oxide Semiconductor). Further, the imaging element 119 may convertan optical image into an electric signal in a portion of the region ofthe photoelectric conversion surface (cutting).

In addition, the imaging element 119 stores an image which is obtainedwhen a photographing instruction is received via the operating unit 180,in the storage medium 200 via the A/D conversion unit 120. On the otherhand, the imaging element 119 outputs images which are continuouslyobtained to the focus adjusting device 191 and the display unit 150 asthrough-the-lens images via the A/D conversion unit 120, in a statewhere an imaging instruction is not received via the operating unit 180.

The A/D conversion unit 120 digitalizes the electric signal converted bythe imaging element 119. In addition, the A/D conversion unit 120outputs the image which is a digital signal to the buffer memory unit130.

The operating unit 180 includes, for example, a power switch, a shutterbutton, multiple selectors (cross keys), or other operation keys. Theoperating unit 180 receives an operation input of a user through anoperation of the user. In addition, the operating unit 180 outputs asignal corresponding to the operation input to the CPU 190.

The image processing unit 140 performs an image process for the imageswhich are temporarily stored in the buffer memory unit 130, by referringto image processing conditions stored in the storage unit 160. Theimages having undergone the image process are stored in the storagemedium 200 via the communication unit 170.

An image obtained by the imaging unit 110, an operation screen, and thelike are displayed. An example of the display unit 150 may include aliquid crystal display. The buffer memory unit 130 temporarily stores animage captured by the imaging unit 110.

A determination condition which is referred to by the CPU 190 at thetime of determination of a scene is stored. In addition, the storageunit 160 stores an imaging condition correlated with each scene which isdetermined through the scene determination.

The CPU 190 controls the imaging unit 110 depending on the set imagingconditions (for example, a diaphragm value, exposure value, and thelike). In addition, the CPU 190 enables the image processing unit 140 toperform an image process for an image as a still screen or a movingimage, based on the “signal corresponding to an operation input” whichis input from the operating unit 180.

The communication unit 170 is connected to the storage medium 200 whichis detachable such as a card memory. The communication unit 170 performsrecording, reading, or deletion of information (image data and the like)for the storage medium 200.

The storage medium 200 is a storage unit which is attachable to anddetachable from the imaging apparatus 100, and stores information (imagedata and the like). In addition, the storage medium 200 may beintegrally formed with the imaging apparatus 100.

Next, the focus adjusting device 191 will be described. The focusadjusting device 191 detects edges of the subject image from an imagegenerated based on the electric signal output by the imaging element119. In addition, the focus adjusting device 191 analyzes colordeviation due to axial chromatic aberration which occurs in the detectededges. Here, the axial chromatic aberration is a characteristic that thefocal length of the lens differs depending on a wavelength (color) ofincident light.

The focus adjusting device 191 detects a defocus (focus deviation)feature amount based on the analysis result of the color deviation. Inaddition, the focus adjusting device 191 generates a driving controlsignal so as to focus a subject image based on the detected defocusfeature amount, and outputs the driving control signal to the barrelcontrol unit 118 of the lens barrel 111 (focus drive).

Here, the defocus feature amount includes a direction index, a defocusamount, a movement amount (hereinafter, referred to as a “focus shiftamount”) necessary for focus adjusting of the AF lens 112, and historythereof. In addition, the direction index is an index indicatingfocalization on a close point side of a subject (hereinafter, referredto as “front focus”), and focalization on a distant point side of thesubject (hereinafter, referred to as “back focus”). Further, the focusshift amount increases as a deviation amount becomes larger.Furthermore, the focus shift amount may be expressed by, for example,the number of steps of a pulse voltage which is output to the lensdriving unit 116 by the barrel control unit 118 in response to a drivingcontrol signal.

As evaluation values for detecting the direction index and the deviationamount, there are three, Edge Difference (hereinafter, referred to as“Ed”), a deviation amount reference value (Width of Subtraction,hereinafter, referred to as “Wd”), and a line spread function(hereinafter, referred to as an “LSF”).

First, the Ed which is one of the evaluation values will be described.FIG. 2 is a diagram illustrating a relationship between a subject at aposition separated from the AF lens 112 by a subject distance, the AFlens 112, an imaging surface (in FIG. 2, shown as “IMAGING SURFACE”) ofthe imaging element 119, and the circle of confusion. The AF lens 112collects light incident from the subject so as to form a subject imageon the photoelectric conversion surface (imaging surface) of the imagingelement 119. Here, the circle of confusion is formed on the imagingsurface of the imaging element 119 due to dot images included in thesubject image.

FIG. 3 shows a positional relationship on the optical axis among a focalpoint of red light (hereinafter, referred to as an “R channel” or “R”),a focal point of green light (hereinafter, referred to as a “G channel”or “G”), and a focal point of blue light (hereinafter, referred to as a“B channel” or “B”), incident to the AF lens 112. In addition, FIG. 3shows that positions (image forming surfaces) of images formed by thelight beams differ from each other depending on the wavelengths of thelight beams. The reason why the image forming surfaces differ dependingon the wavelengths of the light beams is that, for example, refractiveindices of lenses differ depending on the wavelengths of the lightbeams. In addition, the refractive index of a lens is a characteristicunique to the lens and may be different for each lens. Hereinafter, asshown in FIG. 3, a description will be made assuming that the focalpoint of the R channel is located at a position which is farthest fromthe AF lens 112, and the focal point of the B channel is located at aposition closest to the AF lens 112.

FIG. 4 is a diagram illustrating a relationship between a wavelength oflight incident to the AF lens 112 and a position of the imaging surfaceon the optical axis. Here, the transverse axis indicates a wavelength oflight incident to the AF lens 112. In addition, the longitudinal axisindicates a position of the imaging surface, and indicates a positionwhich becomes distant from the AF lens 112 as a value becomes larger. Assuch, the position of the imaging surface becomes distant from the AFlens 112 as the wavelength of incident light becomes longer.

Generally, the AF lens 112 is optically designed such that colordeviation on the image forming surface (imaging surface) in a focalizedstate is the minimum. For this reason, the color deviation becomesgreater in an unfocused state than in a focalized state. In addition, itis known that the color deviation varies depending on the lens position(focus position, zoom position) and other factors. In addition, thecircle of confusion increases in the radius as a defocus (focus blur)amount becomes larger. Further, arrangements of colors of the circle ofconfusion vary depending on the lens position (focus position).

FIGS. 5A, 5B and 5C show the circle of confusion in a front focus state.FIG. 5A shows a relationship between the AF lens 112, the imagingsurface of the imaging element 119, and the circle of confusion. Thelight incident to the AF lens 112 is refracted (dispersed) in an orderof R, G, and B from the outside, is guided to the imaging surface of theimaging element 119, and then forms the circle of confusion. Inaddition, the circle of confusion may be expressed by a dot spreadfunction (hereinafter, referred to as a “PSF”).

FIG. 5B shows the circle of confusion formed on the imaging surface. Thecircle of confusion in a front focus state has an order of R, G, and Bfrom the outside due to the axial chromatic aberration and is thusformed in the following order: red, cyan, and white.

FIG. 5C shows a relationship between a position in the radial directiontoward the outer circumference from the center of the circle ofconfusion and a color component amount (color intensity) using a view(hereinafter, referred to as a “grayscale cross-sectional view”) wherethe longitudinal axis expresses the color component amount. In relationto a gradient (slope) of the color component amount in the radialdirection toward the outer circumference from the center of the circleof confusion, a gradient of R is smoothest and a gradient of B issteepest in a front focus state as shown in FIG. 5C. For this reason,the circle of confusion in a front focus is formed in order of white,cyan, blue, yellow, yellowish red, and red in the radial directiontoward the outer circumference from the center.

FIGS. 6A and 6B show the circle of confusion in a back focus state. FIG.6A shows a relationship between the AF lens 112, the imaging surface ofthe imaging element 119, and the circle of confusion. The light incidentto the AF lens 112 is refracted (dispersed) in an order of B, G, and Rfrom the outside between the focal point on the optical axis and theimaging element 119 due to the axial chromatic aberration, is guided tothe imaging surface of the imaging element 119, and then forms thecircle of confusion on the imaging surface.

FIG. 6B shows the circle of confusion formed on the imaging surface. Thecircle of confusion in a back focus state has an order of B, G, and Rfrom the outer circumference due to the axial chromatic aberration, andis thus formed in order of white, yellow, and blue (bordering) in theradial direction toward the outer circumference from the center.

As such, the formation of the colors in the circle of confusion isdifferent depending on a focalized state and an unfocused state, due tothe axial chromatic aberration. As a result, the focus adjusting device191 (refer to FIG. 1) can detect a focalized state and an unfocusedstate based on the formation of the colors in the circle of confusion.

FIG. 7 shows an example of the captured image. Here, a state where ablack and white edge chart is imaged will be described as an example. Inaddition, for simplicity of description, the description will be madewithout consideration of chromatic aberration of magnification and lensflare, but, even if they are not considered, the spirit of the presentinvention does not vary.

The image 10 (black and white edge chart) formed on the imaging unit 110includes a black region 11 and a white region 12. In addition, the blackregion 11 is rectangle, is located at the center of the white region 12.Here, an axis crossing both the black region 11 and the white region 12in the horizontal direction is set as an axis D. Further, the axis D isan axis perpendicular to a boundary (knife edge) between the blackregion 11 and the white region 12. At the axis D, a boundary where thecolor is changed from white to black is set as an edge position 14.Similarly, at the axis D, a boundary where the color is changed fromblack to white is set as an edge position 13. In addition, a position atthe axis D may be expressed with the pixel units.

FIGS. 8A, 8B and 8C show a color component amount in the vicinity of theedge position 13 (refer to FIG. 7) using grayscale cross-sectionalviews. Here, the longitudinal axis indicates a color component amountfor each of R, G, and B (color channels). In addition, the transverseaxis indicates a position (refer to FIG. 7) at the axis D. In addition,the color component amount of each color channel is expressed by values“0 to 250” (8 bits). The larger the value of the color component amountof each color channel, the deeper the color. In addition, a colorcomponent amount of each color channel may be expressed by values “0 to255” (8 bits).

In addition, a color at a “position at the axis D” where all of thecolor component amounts of R, G, and B are the value “250” is white. Onthe other hand, a color at a “position at the axis D” where all of thecolor component amounts of R, G, and B are the value “0” is black.

FIG. 8A is a grayscale cross-sectional view in a focalized (focus) stateon the G channel. In this state, the R channel is in a front focusstate, and the B channel is in a back focus state. Thereby, the whiteregion 12 side at the edge position 13 becomes green. On the other hand,the black region 11 side at the edge position 13 becomes magenta.

In addition, FIG. 8B is a grayscale cross-sectional view in a focalizedstate on an R channel. In this state, the G channel is in a back focusstate, and a gradient (slope) of the G channel is steeper than agradient of the B channel. In addition, FIG. 8C is a grayscalecross-sectional view in a focalized state on a B channel. In this state,the G channel is in a front focus state, and the gradient (slope) of theG channel is steeper than the gradient of the R channel.

As such, the gradient in the grayscale cross-sectional view is differentfor each color. Therefore, even if a subject image is focused with anycolor channel, the focus adjusting device 191 (refer to FIG. 1)calculates evaluation values from the gradients of the color channels inthe grayscale cross-sectional view. As such, the focus adjusting device191 can detect a focalized state and an unfocused state.

Next, a procedure of calculating the evaluation value Ed will bedescribed. FIGS. 9A and 9B show a color component amount in the vicinityof the edge position 13 (refer to FIG. 7) using the grayscalecross-sectional view. Here, the longitudinal axis indicates the colorcomponent amount for each of R, G, and B (color channel). In addition,the transverse axis indicates a position (refer to FIG. 7) at the axisD. The human eye has high sensitivity for the G channel, and, thus,here, a case where a subject image is focused (focused) using the Gchannel will be described as an example.

FIG. 9A shows a relationship between an edge of the R channel in a frontfocus state, an edge of the R channel in a back focus state, and an edgeof the G channel in a focalized state. In addition, a gradient (slope)of the R channel in a front focus state is smoother than a gradient ofthe G channel in a focalized state. On the other hand, the gradient(slope) of the R channel in a back focus state is steeper than thegradient of the G channel in a focalized state. Hereinafter, in FIG. 9A,a section which is located on the right of an intersection of linesindicating color component amounts of the R channel and the G channeland where the line indicating the G channel is inclined is set as asection “Δ1”. In addition, a section which is located on the left of theintersection of the lines indicating color component amounts and wherethe line indicating the G channel is inclined is set as a section “Δ2”.The length of the sections may be expressed by the number of pixels.

Whether the edge of the R channel is in a front focus state or a backfocus state according to a comparison result of an Ed (hereinafter,referred to as “EdRG”) based on the color component amount of the Rchannel and the color component amount of the G channel and a thresholdvalue is detected (determined) as follows.

EdRG=(Σ(R/G))/(Δ1)>1  Expression 1

EdRG=(Σ(R/G))/(Δ1)<1  Expression 2

Here, the value “1” in Expressions 1 and 2 is a threshold value. Inaddition, (Σ(R/G)) in Expressions 1 and 2 is a sum total which isobtained by adding values over the section “Δ1”, the values beingobtained by dividing the color component amount of the R channel by thecolor component amount of the G channel at the same position in animage. In addition, a value obtained by dividing the sum total by thelength of the section “Δ1” is the “EdRG”. Further, if Expression 1 issatisfied, the EdRG indicates that the edge of the R channel is in aback focus state. On the other hand, if Expression 2 is satisfied, theEdRG indicates that the edge of the R channel is in a front focus state(direction index). In addition, a difference between the EdRG and thethreshold value “1” indicates the defocus amount.

In addition, front focus and back focus may be detected using the“ratio” of the color component amounts in the same manner as Expressions1 and 2 with respect to the section “Δ2”. On the other hand, the SNratio (Signal to Noise ratio) between the color component amount andnoise is small in a section where the color component amount is smallsuch as the section “Δ2”. Therefore, in this case, instead of using the“ratio” of the color component amounts, detecting front focus and backfocus by using a “difference” in the color component amounts isadvantageous in detecting a focalized state.

EdRG=(Σ(G−R))/(Δ2)>0  Expression 3

EdRG=(Σ(G−R))/(Δ2)<0  Expression 4

Here, the value “0” in Expressions 3 and 4 is a threshold value. Inaddition, (Σ(G−R)) in Expressions 3 and 4 is a sum total which isobtained by adding values over the section “Δ2”, the values beingobtained by subtracting a color component amount of the G channel at thesame position in an image from a color component amount of the Rchannel. In addition, a value obtained by dividing the sum total by thelength of the section “Δ2” is the “EdRG”. If Expression 3 is satisfied,the EdRG indicates that the edge of the R channel is in a back focusstate. On the other hand, if Expression 4 is satisfied, the EdRGindicates that the edge of the R channel is a front focus state(direction index). A difference between the EdRG and the threshold value“0” indicates a defocus amount.

Similarly, FIG. 9B shows a relationship between an edge of the B channelin a front focus state, an edge of the B channel in a back focus state,and an edge of the G channel in a focalized state. In addition, agradient (slope) of the B channel in a front focus state is smootherthan a gradient of the G channel in a focalized state. On the otherhand, the gradient (slope) of the B channel in a back focus state issteeper than the gradient of the G channel in a focalized state.Hereinafter, in FIG. 9B, a section which is located on the right of anintersection of lines indicating color component amounts of the Bchannel and the G channel and where the line indicating the G channel isinclined is set as a section “Δ3”. In addition, a section which islocated on the left of the intersection of the lines indicating colorcomponent amounts and where the line indicating the G channel isinclined is set as a section “Δ4”.

Whether the edge of the B channel is in a front focus state or a backfocus state according to a comparison result of an Ed (hereinafter,referred to as “EdBG”) based on a color component amount of the Bchannel and a color component amount of the G channel and a presetthreshold value is detected as follows.

EdBG=(Σ(B/G))/(Δ3)>1  Expression 5

EdBG=(Σ(B/G))/(Δ3)<1  Expression 6

Here, the value “1” in Expressions 5 and 6 is a threshold value. Inaddition, (Σ(B/G)) in Expressions 5 and 6 is a sum total which isobtained by adding values over the section “Δ3”, the values beingobtained by dividing the color component amount of the B channel by thecolor component amount of the G channel at the same position in animage. In addition, a value obtained by dividing the sum total by thelength of the section “Δ3” is the “EdBG”. Further, if Expression 5 issatisfied, the EdBG indicates that the edge of the B channel is in aback focus state. On the other hand, if Expression 6 is satisfied, theEdBG indicates that the edge of the B channel is in a front focus state(direction index). In addition, a difference between the EdBG and thethreshold value “1” indicates a defocus amount.

Front focus and back focus may be detected using a “ratio” of the colorcomponent amounts in the same manner as Expressions 5 and 6 with respectto the section “Δ4”. On the other hand, the SN ratio between a colorcomponent amount and noise is small in a section where the colorcomponent amount is small such as the section “Δ4”. Therefore, in thiscase, instead of using the “ratio” of the color component amounts,detecting front focus and back focus by using a “difference” in thecolor component amounts is advantageous in detecting a focalized state.

EdBG=(Σ(G−B))/(Δ4)>0  Expression 7

EdBG=(Σ(G−B))/(Δ4)<0  Expression 8

The value “0” in Expressions 7 and 8 is a threshold value. In addition,(Σ(G−B)) in Expressions 7 and 8 is a sum total which is obtained byadding values over the section “Δ4”, the values being obtained bysubtracting the color component amount of the G channel at the sameposition in an image from the color component amount of the B channel.In addition, a value obtained by dividing the sum total by the length ofthe section “Δ4” is the “EdRG”. If Expression 7 is satisfied, the EdBGindicates that the edge of the B channel is in a back focus state. Onthe other hand, if Expression 8 is satisfied, the EdBG indicates thatthe edge of the B channel is in a front focus state (direction index). Adifference between the EdBG and the threshold value “0” indicates adefocus amount.

As such, the focus adjusting device 191 (refer to FIG. 1) can detect afocalized state and an unfocused state based on the evaluation value Ed.In addition, the focus adjusting device 191 may detect front focus andback focus using the “ratio” of the color component amounts with respectto the section where the color component amount is small. Further, thefocus adjusting device 191 may detect front focus and back focus using a“difference” in the color component amounts with respect to the sectionwhere the color component amount is large. Although the subject image isfocused with the G channel in the above description, the focus adjustingdevice 191 may detect (determine) a front focus state and a back focusstate by focusing on the subject image with the R channel or the Bchannel.

Next, the Wd which is one of the evaluation values will be described.FIGS. 10A and 10B shows a “color component amount difference” in thevicinity of the edge position 13 (refer to FIG. 7). FIG. 10A shows arelationship between a position at an axis D and a difference betweenthe color component amount of the R channel and the color componentamount of the G channel. Here, the longitudinal axis indicates the colorcomponent amount difference (=R−G) where the color component amount of Gis subtracted from the color component amount of R. In addition, thetransverse axis indicates a position (refer to FIG. 7) at the axis D.Further, the longitudinal axis may indicate the ratio (=R/G) of a colorcomponent amount of R to the color component amount of G instead of thecolor component amount difference.

In addition, in FIGS. 10A and 10B, the solid line indicates arelationship between a color component amount difference (=R−G) in acase where the R channel is in a “back focus state” and a position atthe axis D. The broken line indicates a relationship between the colorcomponent amount difference (=R−G) in a case where the R channel is in a“front focus state” and a position at the axis D.

The Wd is a value (defocus amount reference value) indicating a defocusamount. As shown in FIG. 10A, the Wd is indicated by a distance betweenpeaks of the waveform of the color component amount difference. The Wdis expressed as in Expression 9 using positions X1 and X2 at the axis D.

Wd=|X2−X1|  Expression 9

The positions X1 and X2 are positions where the waveform of the colorcomponent amount difference (color component amount ratio) shows peaksin FIG. 10A. In addition, the intersection of the solid line and thebroken line corresponds to the intersection of the lines of therespective channels in FIG. 9A.

First, in a case where the color component amount difference shows themaximum value “max(R−G)” at the position X2 in FIG. 10A, and the colorcomponent amount difference shows the minimum value “min(R−G)” at theposition X1 (in a case of showing the waveform of the solid line), thepolarity (positive and negative) of the waveform indicates that the Rchannel is in a back focus state (direction index). This is because, ina case where the R channel is in a back focus state, as shown in FIG.9A, the color component amount of the R channel is equal to or more thanthe color component amount of the G channel on the right side of FIG.9A, and the color component amount of the R channel is equal to or lessthan the color component amount of the G channel on the left side ofFIG. 9A.

On the other hand, in a case where the color component amount differenceshows the maximum value “max(R−G)” at the position X1 in FIG. 10A, andthe color component amount difference shows the minimum value “min(R−G)”at the position X2 (in a case of showing the waveform of the brokenline), the polarity (positive and negative) of the waveform indicatesthat the R channel is in a front focus state (direction index). This isbecause, in a case where the R channel is in a front focus state, asshown in FIG. 9A, the color component amount of the R channel is equalto or less than the color component amount of the G channel on the rightside of FIG. 9A, and the color component amount of the R channel isequal to or more than the color component amount of the G channel on theleft side of FIG. 9A.

FIG. 10B shows a waveform in a case of being close to a focalized stateas compared with FIG. 10A. The closer to a focalized state, the smallerthe value of the Wd. In addition, a difference between the maximum value“max(R−G)” of the color component amount difference and the minimumvalue “min(R−G)” of the color component amount difference may varydepending on the “blur degree”.

As such, the focus adjusting device 191 (refer to FIG. 1) can detect afocalized state and an unfocused state based on the evaluation value Wd.In addition, the focus adjusting device 191 can detect a front focusstate and a back focus state based on a polarity of the color componentamount difference.

Next, the LSF which is one of the evaluation values will be described.First, as one of indices for evaluating a lens performance, there is anMTF (Modulation Transfer Function). The MTF expresses to what extent acontrast of a subject can be faithfully reproduced as spatial frequencycharacteristics.

Generally, the MTF of a system is a product of the MTF of the imageforming system (optical system) and the MTF of a sensor (imagingelement). The focus adjusting device can detect (determine) a focalizedstate based on the MTF of the system. For example, the focus adjustingdevice may detect a focalized state by evaluating an MTF of an edge of asubject image. In addition, the focus adjusting device may detect afocalized state in further consideration of image processing procedures(for example, response characteristics of a circuit, demosaic, noisereduction, edge enhancement, and the like) performed in a stagesubsequent to the image forming system and the sensor system.

In a case where light incident to the optical system is incoherent, itis known that the MTF and the LSF are reversible via the Fouriertransform. In addition, the LSF can be easily calculated from edgesincluded in an image (for example, refer to an “ISO12233”specification).

For example, the LSF may be calculated by differentiating an ESF (EdgeSpread Function). In addition, the LSF may be calculated based on adifference between pixel values (for example, color component amounts)of adjacent pixels. In addition, the MTF may be calculated by performinga discrete Fourier transform (DFT) for the LSF.

The focus adjusting device 191 may detect (determine) a focalized statebased on the LSF as an evaluation value in addition to theabove-described Ed and Wd. For example, the focus adjusting device 191may detect a focalized state and an unfocused state (front focus or backfocus) as follows based on a “profile (data) of the LSF for each colorchannel” which is created in advance according to the well-known imageprocessing procedures.

FIGS. 11A, 11B and 11C show an LSF of the G channel according to a “blurdegree” in a focalized state. Here, the LSF is expressed by a standarddeviation σ (statistics) which is one of feature amounts of the LSF. Inaddition, the LSF may be expressed by a full width at half maximum or apeak value which is one of feature amounts of the LSF.

FIG. 11A shows the LSF in a focalized state. FIG. 11B shows the LSF inan unfocused (small blur) state. FIG. 11C shows the LSF in an unfocused(large blur) state in the G channel. From the figures, it is shown thatthe larger the “blur degree” in a focalized state, the greater thestandard deviation a.

FIGS. 12A, 12B and 12C show a “blur degree” and an LSF for each color ina focalized state. FIG. 12A shows an LSF in a focalized state for eachcolor (R, G, and B). In addition, the magnitude correlation for eachcolor channel of the standard deviation σ may be different ascharacteristics unique to a lens. In the following, a description willbe made assuming that, in a standard deviation “σR” of the LSF of the Rchannel in a focalized state, a standard deviation “σG” of the LSF ofthe G channel in a focalized state, and a standard deviation “σB” of theLSF of the B channel in a focalized state, there is a magnitudecorrelation of “σR>σG>σB”.

FIG. 12B shows the LSF in a front focus state for each color (R, G, andB). In addition, a relationship between a position at the axis D and acolor component amount in a front focus state is shown on the rightmostpart of FIG. 12B. In the figure of the rightmost part of FIG. 12B,unlike in FIGS. 8A, 8B and 8C, a case where the G channel is also in afront focus state is shown, and thus all of gradients of R, G, and B aresmooth. In addition, in the figure of the rightmost part of FIG. 12B, inrelation to a gradient (slope) of a color component amount at the edgein a front focus state, it is shown that the gradient of R is thesmoothest, and the gradient of B is the steepest.

As such, since the gradient of R is the smoothest and the gradient of Bis the steepest in a front focus state, in a standard deviation “σRdf”of the LSF of the R channel, a standard deviation “σGdf” of the LSF ofthe G channel, and a standard deviation “σBdf” of the LSF of the Bchannel, in a front focus state, there is a magnitude correlation of“σRdf>σGdf>σBdf”. More specifically, there is a magnitude correlation of“σRdf>K1·σGdf” and “σBdf<K2·σGdf”. Here, “K1=σR/G” and “K2=σB/G”.

FIG. 12C shows the LSF in a back focus state for each color (R, G, andB). In addition, a relationship between a position at the axis D and acolor component amount in a front focus state is shown on the rightmostpart of FIG. 12C. In the figure of the rightmost part of FIG. 12C,unlike in FIGS. 8A, 8B and 8C, a case where the G channel is also in aback focus state is shown, and thus all of gradients of R, G, and B aresmooth. In addition, in the figure of the rightmost part of FIG. 12C, inrelation to a gradient (slope) of a color component amount at the edgein a back focus state, it is shown that the gradient of B is thesmoothest, and the gradient of R is the steepest.

As such, since the gradient of B is the smoothest and the gradient of Ris the steepest in a back focus state, in a standard deviation “σRdb” ofthe LSF of the R channel, a standard deviation “σGdb” of the LSF of theG channel, and a standard deviation “σBdb” of the LSF of the B channel,in a back focus state, there is a magnitude correlation of“σBdb>σGdb>σRdb”. More specifically, there is a magnitude correlation of“σRdb<K1·σGdb” and “σBdb>K2·σGdb”. Here, “K1=σR/G” and “K2=σB/G”.

FIG. 13 shows a relationship between a profile (data) of an LSF for eachcolor (R, G, and B) and a blur degree. The transverse axis indicates ablur degree. In addition, profiles of the LSF are shown in an order ofthe G channel, the R channel, and the B channel from the above. Inaddition, FIG. 13 shows that the number of the subscript of the standarddeviation is large in the LSF where an absolute value (defocus amount)of the blur degree is large.

In addition, the profile of the LSF in a focalized state is shown at thecenter position “0” of the transverse axis. Further, the right side fromthe center position “0” of the transverse axis is a region (front focusregion) where the edge is in a front focus state, and the more distantfrom the center position “0”, the larger the absolute value of the blurdegree. On the other hand, the left side from the center position “0” ofthe transverse axis is a region (back focus region) where the edge is ina back focus state, and the more distant from the center position “0”,the larger the absolute value of the blur degree.

Further, the profile of the LSF for each color channel is created beforethe imaging apparatus 100 images a subject and is stored in advance inthe storage unit 160 for each blur degree. Here, the profile of the LSFfor each color channel is expressed by, for example, a standarddeviation, and is stored in the storage unit 160 in advance. Inaddition, the profile of the LSF may be expressed by a full width athalf maximum or a peak value and be stored in the storage unit 160 inadvance.

In addition, the “blur degree” of the transverse axis may be expressedby a subject distance (depth). Further, the profile of the LSF for eachcolor channel may be correlated with the “blur degree” as well asinformation indicating zoom magnification, a subject distance, and aposition for an angle of view, information indicating a horizontal (H)direction or a vertical (V) direction in a captured image, informationindicating a noise reduction processing method, or informationindicating an edge enhancing method, and be stored in storage unit 160in advance.

FIG. 14 shows a relationship (relative distance, distance function)between a function value having a difference between standard variationsof the LSF and a subject distance. Here, the longitudinal axis indicatesa difference in standard deviations of the LSF. In addition, thetransverse axis indicates a subject distance. Further, the subjectdistance (Depth) is indicated by Expression 10.

Depth=G(F(σRd−σGd)−F1(σBd−σGd))  Expression 10

Here, σRd is a standard deviation of the LSF of the R channel. σGd is astandard deviation of the LSF of the G channel. σBd is a standarddeviation of the LSF of the B channel.

In addition, F(σRd−σGd) is a function having the difference “σRd−σGd”between the standard deviations as a parameter, and has a negative valuein a back focus state and has a positive value in a front focus state.Further, F1(σBd−σGd) is a function having the difference “σBd−σGd”between the standard deviations as a parameter, and is a function havinga positive value in a back focus state and having a negative value in afront focus state.

Furthermore, the function G is a function having“F(σRd−σGd)−F1(σBd−σGd)” as a parameter, and is a function which selectat least one of a value of the function F and a value of the functionF1. In addition, the function G is a function used to calculate thesubject distance (Depth) by selecting a value of the function F in afront focus state and selecting a value of the function F1 in a backfocus state.

In addition, the LSF may be calculated for only one color channel(monochrome). In this case, the focus adjusting device 191 may detect afocalized state and an unfocused state by matching profiles of an imagedLSF of a color channel and the LSF of a single color channel. Forexample, in a case where the imaged LSF conforms (matches) to a profileof an LSF indicating the blur degree “0”, the focus adjusting device 191may determine an edge as being in a focalized state.

As such, the focus adjusting device 191 (refer to FIG. 1) can detect afocalized state or an unfocused state based on a standard deviation or afull width at half maximum of the evaluation value LSF. In addition, thefocus adjusting device 191 can detect a front focus state or a backfocus state by comparing a standard deviation or a full width at halfmaximum of the evaluation value LSF for each color channel.

The above description relates to the evaluation values for detecting adirection index and a defocus amount.

Next, the description of the configuration of the focus adjusting device191 returns. The focus adjusting device 191 (refer to FIG. 1) includesan edge detection unit 192, a distribution detection unit 193, and acontrol unit 194. The edge detection unit 192 detects edges of a subjectimage for each color component (R channel, G channel, and B channel)forming the image which is output by the imaging unit 110.

FIGS. 15A, 15B, 15C and 15D show examples of the captured image, theedge images extracted from the captured image, and the logical productresult of two edge images. FIG. 15A shows an example of a part (partialimage) of the image captured by the imaging unit 110. in FIG. 15A, theframes denoted by the broken lines indicate detection regions for theedge detection unit 192 detecting edges. In addition, the detectionregions may be regions which are spread only toward one side of an edge.Further, a position of the detection region in the partial region may beset in advance.

The edge detection unit 192 (refer to FIG. 1) extracts (detects) an edgeimage (a mask image which masks parts other than the edges) of thesubject image which is captured in the detection regions from thepartial image for each color component. FIG. 15B shows an extracted edgeimage of the R channel. FIG. 15C shows an extracted edge image of the Gchannel. FIG. 15D is an image indicating a calculated result of logicalproduct (AND) of the edge image of the R channel and the edge image ofthe G channel. A procedure of extracting (generating) such images willbe described below.

FIG. 16 is a flowchart illustrating an operation of an edge detectionunit 192. The edge detection unit 192 extracts (generates) an edge imagefor each color channel through a Laplacian filter or the like, from acoarse image (for example, a QVGA resolution) captured by the imagingunit 110 (step Sa1).

The procedure of extracting the edge image for each color channel instep Sa1 will be described more in detail. FIG. 17 is an example of thegrayscale cross-sectional view of a color component amount in thevicinity of the edge position 13 and is a diagram illustrating acalculation example of using a Laplacian filter. As an example, a caseof extracting (generating) an edge extending in the vertical directionof the image will be described.

Here, it is assumed that pixels are arranged in an order of a pixel A, apixel B, and a pixel C in the horizontal direction perpendicular to theedge. In addition, in the respective pixels, it is assumed that thepixel A is formed by “a color component amount R1=value “200”, a colorcomponent amount G1=value “220”, and a color component amount B1=value“10”, the pixel B is formed by “a color component amount R2=value “220”,a color component amount G2=value “230”, and a color component amountB2=value “10”, and the pixel C is formed by “a color component amountR3=value “220”, a color component amount G3=value “250”, and a colorcomponent amount B3=value “10”. In addition, a threshold value Ek1 isset to a value “100”. Further, these values are an example.

The edge detection unit 192 adds a value obtained by multiplying eachcolor component amount by a coefficient of the Laplacian filter to eachcolor component. In addition, the edge detection unit 192 compares theaddition result with the preset threshold value Ek1 for each colorcomponent.

Specifically, the edge detection unit 192 compares the addition result“(−1)×R1+8×R2+(−1)×R3” for the R channel with the threshold value Ek1.Since the addition result for the R channel is greater than thethreshold value Ek1, the edge detection unit 192 uses the edge of the Rchannel as an effective edge and generates an edge image of the Rchannel.

Specifically, the edge detection unit 192 compares the addition result“(−1)×G1+8×G2+(−1)×G3” for the G channel with the threshold value Ek1.Since the addition result for the G channel is greater than thethreshold value Ek1, the edge detection unit 192 uses the edge of the Gchannel as an effective edge and generates an edge image of the Gchannel. Similarly, the edge detection unit 192 compares the additionresult “(−1)×B1+8×B2+(−1)×B3” for the B channel with the threshold valueEk1. Since the addition result for the B channel is equal to or smallerthan the threshold value Ek1, the edge detection unit 192 does not usethe edge of the B channel as an effective edge and does not generate anedge image of the B channel.

In addition, the edge detection unit 192 sets the color component amountof the R channel in the edge image to the same level as the colorcomponent amount of the G channel. For example, the edge detection unit192 uses a value obtained by dividing the color component amount of theG channel by the color component amount of the R channel, as a gain(correction magnification). In addition, the edge detection unit 192multiplies the color component amount of the R channel by the gain. Inthis way, the edge detection unit 192 generates edge images (refer toFIGS. 15B and 15C).

Returning to FIG. 16, the description of the operation of the edgedetection unit 192 is continued. The edge detection unit 192 extractsedges (edge pair) of two or more colors which are correlated for aposition. Specifically, the edge detection unit 192 extracts a colorcomponent amount of the edge of the R channel and a color componentamount of the edge of the G channel, for the same pixel. In addition,the edge detection unit 192 designates a color component amount logicalproduct (AND) of the edge pair as a common edge image (step Sa2).

The edge detection unit 192 may extract a logical product of colorcomponent amounts of edges (edge pair, color pair) of the R channel andthe G channel adjacent within “±1 pixel” as a common edge, even if theedges are in the same pixel (generation of a thick line). In additionthe edge detection unit 192 may extract a common edge based on a logicalproduct of color component amounts of edges equal to or more than apreset length L.

In addition, the edge detection unit 192 stores common edges of the Rchannel and the G channel forming the common edges in the storage unit160 (step Sa3). In this way, the edge detection unit 192 generates anedge image (refer to FIG. 15D) where a common edge between the edge ofthe R channel and the edge of the G channel.

The imaging unit 110 divides the captured image into lattice-shapedblocks. In addition, the distribution detection unit 193 detects afocalized state and an unfocused state (a direction index and a defocusamount) for each of the divided blocks.

FIG. 18 shows an example of the depth map. Here, the depth map is a mapwhich shows distributions of a focalized state and an unfocused state(blur degree) in the captured image with the block (partial image)units.

In the following, as an example, a description will be made assumingthat an image captured by the imaging unit 110 is divided into “4×4”blocks, and an edge image in one block of them is the edge imageexemplified in FIG. 15D.

The distribution detection unit 193 detects distributions of a focalizedstate and an unfocused state of the common edge based on at least one ofan Ed, a Wd or an LSF of the common edge extracted in the edge image.The distribution detection unit 193 detects a focalized state and anunfocused state (blur degree) of the common edges as follows based on anEd of the three common edges extracted in the edge image of FIG. 15D.

The lengths of the three common edges extracted in the edge image ofFIG. 15D are respectively indicated by L1, L2, and L3. First, thedistribution detection unit 193 calculates Expression 11.

$\begin{matrix}{\Sigma\left( {L\left( {\left( {{\Sigma \left( {R/G} \right)}/\Delta} \right) = {{L\; 1\left( {\left( {\Sigma \left( {R\; {1/G}\; 1} \right)} \right)/\Delta} \right)} + {L\; 2\left( {\left( {\Sigma \left( {R\; {2/G}\; 2} \right)} \right)/\Delta} \right)} + {L\; 3\left( {\left( {\Sigma \left( {R\; {3/G}\; 3} \right)} \right)/\Delta} \right)}}} \right.} \right.} & {{Expression}\mspace{14mu} 11}\end{matrix}$

Here, “Δ” is a length of a section perpendicular to the edge. Forexample, in a case of the grayscale cross-sectional view exemplified inFIG. 9A, “Δ” is the length of the section Δ1 or Δ2. In addition, thedistribution detection unit 193 sets a values obtained by dividingExpression 11 “Σ(L((Σ(R/G)/A)” by “L1+L2+L3” as the EdRG.

In addition, as described in the evaluation value Ed, in a case whereExpression 1 is satisfied, the distribution detection unit 193determines that the common edges are in a back focus state. On the otherhand, in a case where Expression 2 is satisfied, the distributiondetection unit 193 determines that the common edges are in a front focusstate.

In the depth map exemplified in FIG. 18, the value surrounded by therectangular frame indicates a “blur degree” in the block. In addition,the larger the absolute value of the blur degree, the larger the defocusamount, which indicates an unfocused state. In addition, the blockhaving a negative value indicates that an edge imaged in the block is ina back focus state. Conversely, the block having a positive valueindicates that an edge imaged in the block is in a front focus state.

The distribution detection unit 193 may subdivide one of the blocksdivided into “4×4”, into small blocks (segment) of “2×2”. In addition,the distribution detection unit 193 may detect a focalized state and anunfocused state of the common edges for the small blocks based on ahistogram of the blur degree. For example, the distribution detectionunit 193 may interpolate a “blur degree” of the small block between avalue “2” and a value “8” using the histogram so as to generate a value“4”. The interpolated value is an example. In addition, in a case wherean effective edge of which a color component amount (power) issufficient is not detected, an undetectable block where a directionindex and a defocus amount cannot be detected may be left in the depthmap.

In this way, the distribution detection unit 193 detects distributionsof a focalized state and an unfocused state of the common edges based onthe Ed, thereby creating a depth map. Similarly, the distributiondetection unit 193 may detect a focalized state and an unfocused stateof the common edges based on the above-described Wd, thereby creating adepth map.

In addition, the distribution detection unit 193 may detect a focalizedstate and an unfocused state of the common edges based on theabove-described LSF, thereby creating the depth map. Specifically, thedistribution detection unit 193 divides an image captured in the imagingunit 110 into lattice-shaped blocks, and extracts an edge image for eachblock (refer to FIG. 15D). In addition, the distribution detection unit193 calculates an LSF of an edge extracted in the edge image accordingto the procedure described in the evaluation value LSF. Further, thedistribution detection unit 193 detects a “blur degree” for each blockaccording to the procedure described in the evaluation value LSF basedon a profile of the LSF and an imaged LSF of the edge, thereby creatinga depth map.

In addition, the distribution detection unit 193 may detectdistributions of a focalized state and an unfocused state of the commonedges by combining the evaluation values Ed, Wd and LSF with each other.For example, the distribution detection unit 193 may detectdistributions of a focalized state and an unfocused state of the commonedges based on the Ed, and then supplementarily detect distributions ofa focalized state and an unfocused state of the common edges based onthe LSF for a block where an effective edge is not detected.

The control unit 194 selects blocks which capture the subject image fromthe distributions based on the distributions of a focalized state and anunfocused state detected by the distribution detection unit 193. Inaddition, the control unit 194 moves the AF lens 112 so as to focus onthe edges of the subject image. Further, the control unit 194 determineswhether or not a photographing instruction for storing an image capturedby the imaging unit 110 in the storage medium 200 is output from theoperating unit 180 to the CPU 190. Furthermore, the control unit 194determines whether or not a focus instruction for focusing on a subjectimage is output from the operating unit 180 to the CPU 190. The controlunit 194 changes procedures of the focus drive based on thedetermination as described later.

Next, an operation of the imaging apparatus 100 will be described mainlybased on an operation of the focus adjusting device 191. FIG. 19 is aflowchart illustrating a procedure of the focus drive of the focusadjusting device 191 in a tracking operation of tracking a subject.First, the CPU 190 of the imaging apparatus 100 sets the focus adjustingdevice 191 in a tracking mode so as to execute a tracking operation oftracking a subject image (step S1). In addition, the CPU 190 designates(sets) the subject image which is a tracking target in the focusadjusting device 191 (step S2). These settings may be executed based onan operation input received by the operating unit 180.

In addition, the control unit 194 of the focus adjusting device 191executes a first focus drive (step S3). A focusing method in the focusdrive may be appropriately selected. For example, the focusing methodmay be a contrast AF (Auto Focus) method, or may be other focusingmethods (phase difference detection method and the like). The imagingelement 119 further converts an optical image formed on thephotoelectric conversion surface into an electric signal according tothe focusing method. In addition, the imaging element 119 outputs theelectric signal obtained here to the A/D conversion unit 120 as athrough-the-lens image (step S4). A resolution of the through-the-lensimage output from the imaging element 119 may be reduced by decimating aportion of the data.

Next, the imaging apparatus 100 and the focus adjusting device 191process steps S5 to S7 and steps S8 to S16 described below in parallel.In addition, the imaging apparatus 100 and the focus adjusting device191 may repeatedly execute steps S4 to S17.

The CPU 190 controls the imaging unit 110 so as to execute an automaticexposure (AE) process and a debayer (color interpolation) process. Inaddition, the image processing unit 140 acquires the through-the-lensimage and executes an image process (step S5). For example, the imageprocessing unit 140 executes a white balance (WB) adjustment process, anoise reduction (NR) process, a magnification chromatic aberrationcorrection process, and the like, as image processes, for the acquiredthrough-the-lens image.

In addition, the image processing unit 140 searches for the trackingtarget imaged in a tracking region of the acquired through-the-lensimage through an image matching process or the like (step S6). Inaddition, the image processing unit 140 superimposes a tracking rangeindicating the tracking region on the through-the-lens image so as to bedisplayed on the display unit 150 (step S7).

Meanwhile, the edge detection unit 192 of the focus adjusting device 191extracts edges imaged in a tracking region of an initial window size setin advance (refer to each figure of FIGS. 15A, 15B, 15C, and 15D) (stepS8). In addition, the edge detection unit 192 analyzes edgecharacteristics according to the procedure described with reference toFIG. 16 (step S9).

Further, the edge detection unit 192 determines whether or not there areedges which are tracking targets (step S10). If there is no edge (stepS10-NO), the process of the edge detection unit 192 returns to step S8.On the other hand, if there is an edge (step S10-YES), the edgedetection unit 192 sets the edge as a tracking target (step S11). Here,the edge detection unit 192 may set an edge of which power is strong (acolor component amount is large) as a tracking target.

In addition, the CPU 190 controls the imaging unit 110 so as to increasethe resolution of the through-the-lens image output from the imagingelement 119. Here, there are cases where the resolution of thethrough-the-lens image is increased as a result of decimation of aportion of data of the through-the-lens image. In addition, the CPU 190controls the imaging unit 110 such that a portion of the optical imageformed on the photoelectric conversion surface of the imaging element119 is cut, and an image thereof is converted into an electric signalwhich is output to the A/D conversion unit 120 from the imaging element119 (step S12).

In addition, the edge detection unit 192 analyzes edge characteristicsand stores evaluation values in the storage unit 160 (step S13). In stepS13, the edge detection unit 192 executes a procedure shown in FIG. 20.FIG. 20 is a flowchart illustrating an analysis procedure of the edgecharacteristics. The edge detection unit 192 calculates a gain(magnification) based on an edge image of the R channel and an edgeimage of the G channel, and multiplies the edge of the R channel by thegain (refer to FIG. 17 and the like) (step Sb1).

In addition, the edge detection unit 192 transforms an edge extending ina tilted manner with respect to the image so as to be horizontal withrespect to the image by executing affine transform or cosine correction(step Sb2). In addition, the distribution detection unit 193 of thefocus adjusting device 191 calculates an evaluation value for eachlattice-shaped block in the image captured by the imaging unit 110 basedon the section Δ in the vicinity of the edge (for example, refer toExpression 1).

In addition, the distribution detection unit 193 calculates eachevaluation value for each edge (refer to FIG. 15D) extracted by the edgedetection unit 192, and sets a value obtained by averaging thecalculated evaluation values with the length of the edge, as an Ed (forexample, refer to Expression 11) (step Sb4). Here, the distributiondetection unit 193 calculates at least one of the Ed, the Wd, and theLSF as evaluation values. In addition, the distribution detection unit193 stores the evaluation values, edge pairs (color pair) for each colorchannel, color component amounts (color intensity), and gains, as theedge characteristics, in the storage unit 160. The distributiondetection unit 193 creates a depth map (refer to FIG. 18) in the mannerdescribed above.

Returning to FIG. 19, the description of the procedure of the focusdrive of the focus adjusting device 191 is continued. The distributiondetection unit 193 determines in which one of a focalized state, a frontfocus state, or a back focus state the edge is, based on an evaluationvalue of the block capturing the edge of the subject image which is setas a tracking target in step S2 (for example, refer to Expressions 1 to4) (step S14).

In addition, the distribution detection unit 193 determines whether ornot the evaluation value Ed exceeds a predefined first threshold value.Further, the distribution detection unit 193 determines whether or not adifference between the previous Ed and the present Ed exceeds apredefined second threshold value (step S15).

If the evaluation value Ed exceeds the predefined first threshold value,or the difference between the previous Ed and the present Ed exceeds thepredefined second threshold value (step S15-YES), it can be determinedthat a defocus amount is increased. Therefore, the control unit 194restricts a movement direction of the AF lens 112 and executes contrastscanning

Next, a procedure where the control unit 194 restricts a movementdirection of the AF lens 112 will be described. FIG. 21 shows a scanningregion when a front focus state is determined and a scanning region whena back focus state is determined. The middle part of FIG. 21 shows that,in “normal contrast scanning”, the AF lens 112 moves while performingcontrast scanning for the entire range of a predefined movable range(close to infinite) inside the lens barrel 111. The “normal contrastscanning” is performed regardless of a determination result of a frontfocus state and a back focus state.

In contrast to the “normal contrast scanning” shown in the middle partof FIG. 21, the lower part of FIG. 21 shows a movement in contrastscanning performed based on a determination result of a front focusstate and a back focus state. According to the lower part of FIG. 21,the AF lens 112 moves while performing contrast scanning for a portionof the region of the predefined movable range (close to infinite) insidethe lens barrel 111.

If it is determined that the edge of the subject image set as a trackingtarget is in a front focus state, the control unit 194 restricts amovement direction of the AF lens 112 to a region on the close side fromthe present position of the AF lens 112 and performs contrast scanning.On the other hand, If it is determined that the edge of the subjectimage set as a tracking target is in a back focus state, the controlunit 194 restricts a movement direction of the AF lens 112 to a regionon the infinite side from the present position of the AF lens 112 andperforms contrast scanning

As such, since a movement direction of the AF lens 112 is restricted,the focus adjusting device 191 enables the subject image to be rapidlyfocused as compared with the normal contrast scanning

Returning to FIG. 19, the description of the procedure of the focusdrive of the focus adjusting device 191 is continued. The control unit194 moves the AF lens 112 to a focus position which is detected throughthe contrast scanning performed based on the determination result of afront focus state and a back focus state in order to focus on the edgeof the subject image (step S16).

In step S15, in a case where the evaluation value Ed does not exceed thepredefined first threshold value and the difference between the previousEd and the present Ed is not increased more than the predefined secondthreshold value (step S15-NO), the process of the control unit 194proceeds to step S17. In addition, the process of the control unit 194executes step S16 and then proceeds to step S17.

The control unit 194 determines whether or not a photographinginstruction for storing an image captured by the imaging unit 110 in thestorage medium 200 is output from the operating unit 180 to the CPU 190(step S17). For example, a user operates the shutter button of theoperating unit 180 and thereby the photographing instruction is outputto the CPU 190 from the operating unit 180.

In a case where the photographing instruction is not output to the CPU190 from the operating unit 180 (step S17-NO), the process of thecontrol unit 194 returns to step S4. On the other hand, in a case wherethe photographing instruction is output to the CPU 190 from theoperating unit 180 (step S17-YES), the process of the control unit 194proceeds to step S18.

Next, the control unit 194 executes AF for photographing (step S18).FIGS. 22A, 22B and 22C show a move of a lens position in hill-climbingcontrast scanning during AF for photographing. In addition, FIG. 22Ashows an example of the relationship between a position of the AF lens112 and a contrast value in the contrast scanning Here, a region on theclose side from the present position of the lens is referred to as aclose side region. A region on the infinite side from the presentposition of the lens is referred to as an infinite side region.

In a case where the AF lens 112 is moved from a start position (forexample, a position in the close side region) to an end position (forexample, a position in the infinite side region) in the predefined lensmovable range inside the lens barrel 111 under the control of thecontrol unit 194, a lens position where the contrast value shows a peakvalue is set as a focus position. If the AF lens 112 is located at thefocus position, the edge of the subject image is in a focalized state.

FIG. 22B shows a movement of a position of the AF lens 112 in the normalcontrast scanning (refer to the middle part of FIG. 21). In the normalcontrast scanning, the control unit 194 moves the AF lens 112 whileperforming contrast scanning for the entire region of the predefinedlens movable lens inside the lens barrel 111.

The control unit 194 moves the AF lens 112 from a lens position at thetime point t1 to the start position of the contrast scanning during thetime t1 to t2 (initial position driving). In addition, the control unit194 moves the AF lens 112 from the start position to the end positionduring the time t2 to t3 while performing the contrast scanning(scanning driving). In addition, the control unit 194 moves the AF lens112 to the focus position where a contrast value shows a peak during thetime t3 a to t4 a (focus position driving).

As such, in the normal contrast scanning, the control unit 194 performscontrast scanning for the entire region of the lens movable range. Forthis reason, it takes time to move the AF lens 112 to a focus position.

On the other hand, FIG. 22C shows a movement of a position of the AFlens 112 in the contrast scanning (refer to the lower part of FIG. 21)based on determination of a front focus state and a back focus statedepending on an evaluation value. The control unit 194 moves the AF lens112 while performing contrast scanning for a partial region of thepredefined lens movable lens inside the lens barrel 111 based ondetermination of a front focus state and a back focus state.

In addition, the distribution detection unit 193 detects distributionsof a focalized state and an unfocused state of the common edges. Here,it is assumed that the edge of the subject image which is a trackingtarget is in a front focus state, and the distribution detection unit193 performs determination until the time point t1. The control unit 194moves the AF lens 112 from a lens position at the time point t1 to thestart position of the contrast scanning during the time t1 to t2(initial position driving).

Since it is determined that the edge of the subject image is in a frontfocus state, the control unit 194 predicts that a focus position is inthe close side region. In addition, the control unit 194 moves the AFlens 112 from the start position to the “lens position at the time pointt1” during the time t2 to t3 b while performing the contrast scanning(scanning driving). In addition, the control unit 194 moves the AF lens112 to the focus position where a contrast value shows a peak during thetime t3 a to t4 b (focus position driving).

Here, the control unit 194 may finish the focus drive at a time pointwhen the evaluation value Ed converges in a predefined range in thedriving scanning. In addition, the control unit 194 may detects a frontfocus state based on the LSF as described above and finish the focusdrive in the scanning driving.

Further, the control unit 194 passes a position where a contrast valueshows a peak while performing contrast scanning in the scanning driving,and thereby calculates a focus position through interpolation. In a casewhere a focus position is calculated through the interpolation, there ishigh possibility that a position may be more accurately detected than ina case where it is not calculated through the interpolation. Further,the control unit 194 moves the AF lens 112 to the focus positioncalculated through the interpolation (focus position driving).

Returning to FIG. 19, the description of the focus drive is continued.The CPU 190 controls the imaging unit 110 so as to exposure the edgewhich is in a focalized state through AF for photographing. In addition,the CPU 190 stores (records) the image captured by the imaging unit 110in the storage medium 200 via the communication unit 170 (step S19).

As described above, the focus adjusting device 191 includes the edgedetection unit 192 which detects an edge of a subject image for eachcolor component forming an image including the subject image which isincident from the lens barrel 111 having the AF lens 112 for performingfocus adjustment. In addition, the focus adjusting device 191 furtherincludes the distribution detection unit 193 which detects distributionsof a focalized state and an unfocused state in an image based on theedge for each color component detected by the edge detection unit 192.Further, the focus adjusting device 191 further includes the controlunit 194 which moves the AF lens 112 so as to focus on the subject imagebased on the distributions (depth map) detected by the distributiondetection unit 193.

Thereby, the control unit 194 moves the AF lens 112 only in the closeside region in the contrast scanning based on the determination that thecommon edges are in a front focus state. As a result, the focus drivefinishes in a short time as compared with a case of performing thenormal contrast scanning. Similarly, the control unit 194 moves the AFlens 112 only in the infinite side region in the contrast scanning basedon the determination that the common edges are in a back focus state. Asa result, the focus drive finishes in a short time as compared with acase of performing the normal contrast scanning

Therefore, the focus adjusting device 191 enables a subject image whichhas never been focused to be rapidly focused.

In addition, the distribution detection unit 193 detects distributionsof a focalized state and an unfocused state based on a gradient of acolor component amount of the edge which is detected for each colorcomponent by the edge detection unit 192. Thereby, the focus adjustingdevice 191 enables the subject image to be rapidly focused based on thegradient of the color component amount of the edge.

In addition, the distribution detection unit 193 detects a directionindex indicating focalization on a close point side from a subject or adistant point side from the subject based on a ratio of or a differencebetween color component amounts of the edges which are edges for eachcolor component detected by the edge detection unit 192. In addition,the distribution detection unit 193 also detects a defocus amount.Thereby, the focus adjusting device 191 enables the subject image to berapidly focused without increasing the calculation load based on theratio of or the difference between the color component amounts of theedges.

Further, the distribution detection unit 193 detects a defocus amountbased on a distance between peaks of the ratio of the difference betweenthe color component amounts of the edges which are detected for colorcomponent by the edge detection unit 192. Thereby, the focus adjustingdevice 191 enables the subject image to be rapidly focused withoutincreasing the calculation load based on the distance between peaks ofthe ratio of or the difference between the color component amounts ofthe edges.

(As to Case of Executing Focus Drive Based on Evaluation Value LSF)

FIG. 23 is a flowchart illustrating a procedure of the focus drive basedon the LSF. The CPU 190 of the imaging apparatus 100 sets a region(central area region) at the center of the through-the-lens image as aregion where an edge is focused (AF) (step Sd1). This setting may beexecuted based on an operation input of a user received by the operatingunit 180.

The CPU 190 controls the imaging unit 110 so as to perform exposurecontrol (step Sd2). In addition, the image processing unit 140 acquiresa through-the-lens image (step Sd3). Further, the CPU 190 controls theimaging unit 110 so as to execute a debayer (color interpolation)process.

In addition, the image processing unit 140 executes image processes forthe acquired through-the-lens image. For example, the image processingunit 140 executes a white balance (WB) adjustment process, a noisereduction (NR) process, a magnification chromatic aberration correctionprocess, and the like, as the image processes, for the acquiredthrough-the-lens image (step Sd4). Here, the through-the-lens imageacquired by the image processing unit 140 may be an image of which aresolution is reduced by decimating an electric signal output by theimaging element 119. In this way, an amount of the image processesexecuted by the image processing unit 140 can be reduced. In addition,the image processing unit 140 displays the through-the-lens image havingundergone the image processes on the display unit 150 (step Sd5).

The CPU 190 determines whether or not a focus instruction is input tothe CPU 190 (step Sd6). If the focus instruction is not input (stepSd6-NO), the process of the CPU 190 returns to step Sd3. If the focusinstruction is input (step Sd6-YES), the process of the CPU 190 proceedsto step Sd7.

The image processing unit 140 acquires the through-the-lens image at thefull resolution (step Sd7). In addition, the CPU 190 controls theimaging unit 110 so as to execute a debayer (color interpolation)process. In addition, the image processing unit 140 executes imageprocesses for the acquired through-the-lens image. For example, theimage processing unit 140 executes a white balance adjustment processand a noise reduction process for the acquired through-the-lens image(step Sd8).

The edge detection unit 192 extracts an edge image for each colorchannel through a Laplacian filter in the central area region (stepSb9). In addition, the edge detection unit 192 extracts effective edgesof which a color component amount (edge power) is equal to or more thana specific amount (step Sb10). Further, the edge detection unit 192calculates an LSF of the extracted edges with respect to the horizontaldirection of the image (step Sd11).

The edge detection unit 192 detects a positional correlation for theedges of each color channel (step Sb12). The edge detection unit 192selects edges of channels of two or more colors having the positionalcorrelation so as to extract an edge image (step Sb13). Here, a casewhere an edge of the R channel and an edge of the G channel will bedescribed as an example. The edge detection unit 192 performs thick linegeneration for the edge of the extracted edge image and creates a maskimage (refer to FIG. 15D) where parts other than the edge are masked(step Sd14).

The distribution detection unit 193 divides the mask image into blocksof “8×8”. In addition, the distribution detection unit 193 calculates anaverage value of the LSF around one pixel by referring to the LSF ofeach block (step Sd15). In addition, the distribution detection unit 193may execute labeling of the subject image based on the calculatedaverage value. If there is a plurality of labels in the central arearegion, the distribution detection unit 193 determines that a pluralityof subject distances is in the central area region (step Sd16).

The control unit 194 calculates a driving direction and a defocus amount(a start position of contrast scanning) of the AF lens 112 based on theedge of the R channel and the edge of the G channel (step Sd17). Thecontrol unit 194 moves the AF lens 112 to the start position (forexample, if in a front focus state, a position in the close sideregion). In addition, the control unit 194 executes the contrastscanning while calculating an LSF. The control unit 194 temporarilypasses a position (refer to FIG. 13) where the LSF indicates a focalizedstate while performing the contrast scanning, and then stops thescanning driving (step Sd18).

In addition, the control unit 194 calculates a focus position throughinterpolation, and moves the AF lens 112 to the calculated focusposition (step Sd19). In addition, the image processing unit 140repeatedly acquires a through-the-lens image (step Sd20). Further, theimage processing unit 140 displays the through-the-lens image on thedisplay unit 150 (step Sd21). The control unit 194 may set a regionwider than the central area region as a region (AF region) where an edgeis focused.

As described above, the distribution detection unit 193 detects adirection index indicating focalization on a close point side from asubject or a distant point side from the subject based on a line spreadfunction (LSF) corresponding to the edges which are edges detected foreach color component by the edge detection unit 192. In addition, thedistribution detection unit 193 also detects a defocus amount. Thereby,the focus adjusting device 191 enables the subject image to be rapidlyfocused even in a case where the subject image is monochrome, a casewhere the area of the subject image is small, or a spatial frequency ofthe subject image is low (content resistance).

In addition, the distribution detection unit 193 detects a defocusamount based on a standard deviation or a full width at half maximum ofa line spread function (LSF) corresponding to the edges which aredetected for each color component by the edge detection unit 192.Thereby, the focus adjusting device 191 enables the subject image to berapidly focused based on the standard deviation or the full width athalf maximum of the edges.

The focus adjusting device 191 includes the edge detection unit 192which detects an edge of a subject image for each color componentforming an image including the subject image which is incident from thelens barrel 111 having the AF lens 112 for performing focus adjustment.In addition, the focus adjusting device 191 further includes thedistribution detection unit 193 which calculates a line spread function(LSF) of the edge detected for each color component by the edgedetection unit 192. Further, the focus adjusting device 191 furtherincludes the control unit 194 which moves the AF lens 112 so as to focuson the subject image based on the line spread function.

Thereby, the focus adjusting device 191 enables a subject image to berapidly focused even in a case where the subject image is monochrome, acase where the area of the subject image is small, or a spatialfrequency of the subject image is low.

Second Embodiment

The second embodiment of the present invention will be described withreference to the drawings. The second embodiment is different from thefirst embodiment in that the distribution detection unit 193 predicts afocus position (defocus amount) based on a “defocus-driving pulse table”described later. Hereinafter, only differences between the secondembodiment and the first embodiment will be described.

FIG. 24 shows an example of the “defocus-driving pulse table”. As items(references) of the defocus-driving pulse table, there are a lensposition, an evaluation value, the number of steps of driving pulses ina front focus state (hereinafter, referred to as “the number of steps ina front focus state”), and the number of steps of driving pulses in aback focus state (hereinafter, referred to as “the number of steps in aback focus state”). Here, the evaluation value may be any defocus amountbased on the Ed, the Wd, and the LSF.

In addition, the number of steps of driving pulses is driving pulseswhich are output to the lens driving unit 116 by the barrel control unit118 in response to a driving control signal. The number of steps ofdriving pulses is set in advance based on a structure of the barrelcontrol unit 118. In addition, the number of steps in a front focusstate indicates the number of steps necessary to be moved from a presentlens position in a front focus state to a focus position in a focalizedstate. Similarly, the number of steps in a back focus state indicatesthe number of steps necessary to be moved from a present lens positionin a back focus state to a focus position in a focalized state. Further,the defocus-driving pulse table may be stored in the storage unit 160.

The distribution detection unit 193 periodically an evaluation value(defocus amount). In addition, the distribution detection unit 193registers the detected evaluation value and a lens position whendetected in the item of the “evaluation value” of the defocus-drivingpulse table. In addition, the distribution detection unit 193 registersthe “number of steps in a front focus state” and the “number of steps ina back focus state” in the defocus-driving pulse table. The “number ofsteps in a front focus state” and the “number of steps in a back focusstate” are set in advance based on a structure of the barrel controlunit 118. The distribution detection unit 193 repeatedly performs theregistration operation and thereby creates history.

Next, a description will be made of a procedure where the control unit194 restricts a movement direction of the AF lens 112 and a contrastscanning region based on the defocus-driving pulse table.

FIG. 25 shows a contrast scanning region when front focus and a degreethereof are determined. In addition, FIG. 25 also shows a contrastscanning region when back focus and a degree thereof are determined. Ina case where it is determined that an edge of a subject image set as atracking target is in a front focus state, the control unit 194 sets aprediction range of a focus position in the close side region based onthe defocus-driving pulse table (refer to FIG. 24).

For example, it is assumed that the lens position (focus position)output from the barrel control unit 118 is a value “1”, the evaluationvalue (defocus amount) detected by the distribution detection unit 193is a value “1”, and the direction index detected by the distributiondetection unit 193 indicates a front focus state.

In this case, the control unit 194 refers to whether or not the numberof steps in front focus corresponding to the lens position “1” and theevaluation value “1” is registered in the defocus-driving pulse table(refer to FIG. 24). In the example shown in FIG. 24, “the number ofsteps in front focus” corresponding to the lens position “1” and theevaluation value “1” is registered. Therefore, the control unit 194acquires the number “23” of steps in front focus from thedefocus-driving pulse table.

Thereby, the control unit 194 predicts that there is a focus position ata position where the AF lens 112 is moved in a close direction from thepresent lens position by a movement amount corresponding to the number“23” of steps of driving pulses. In addition, the control unit 194 setsa prediction range of a predetermined length with respect to thepredicted position. Further, the control unit 194 performs contrastscanning in the restricted prediction range. In addition, a safetycoefficient is added to the length of the prediction range which may befurther lengthened.

Similarly, in a case as well where it is determined that the edge of thesubject image set as a tracking target is in a “back focus state”, thecontrol unit 194 sets a prediction range of a focus position withrespect to the focus position which is in the infinite side region basedon the defocus-driving pulse table (refer to FIG. 24). Further, thecontrol unit 194 performs contrast scanning in the restricted predictionrange.

FIG. 26 is a flowchart illustrating a procedure of the focus drive ofthe focus adjusting device 191 in a tracking operation of tracking asubject. Steps Sc1 to Sc15 of FIG. 26 are the same as steps S1 to S15 ofFIG. 19. Steps Sc18 to Sc20 of FIG. 26 are the same as steps S17 to S19of FIG. 19. In addition, steps Sc16 and Sc17 of FIG. 26 indicateparallel processes.

If the evaluation value Ed exceeds the predefined first threshold value,or the difference between the previous Ed and the present Ed exceeds thepredefined second threshold value (step Sc15-YES), the control unit 194restricts a movement direction of the AF lens 112. In addition, thecontrol unit 194 further sets a prediction range of a focus position byusing FIG. 24 and referring to the defocus-driving pulse table (stepSc16).

The control unit 194 restricts a movement direction of the AF lens 112,further restricts it to the prediction range of a focus position, andperforms contrast scanning (small scanning) In addition, the controlunit 194 moves the AF lens 112 to the focus position so as to focus onthe edge of the subject image (step Sc17). Further, the process of thecontrol unit 194 proceeds to step Sc18. The subsequent steps are thesame as in the first embodiment.

As such, a range where the contrast scanning is performed is restrictedto a prediction range of a focus position. As a result, the focusadjusting device 191 enables a subject image to be more rapidly focusedthan in the normal contrast scanning

FIG. 27 shows an example of the focus drive in the contrast scanningbased on the “defocus-driving pulse table”. Hereinafter, a case wherethe control unit 194 executes the focus drive based on the evaluationvalue Ed will be described as an example. Each transverse axis of FIG.27 indicates the number of frames of a through-the-lens image capturedby the imaging unit 110. The longitudinal axis of the upper part of FIG.27 indicates an Ed of a common edge. Here, an Ed at the number f1 offrames is referred to as an “Ed value (initial)”. In addition, thelongitudinal axis of the lower part of FIG. 27 indicates a lensposition.

In a case where a specific time has elapsed from the contrast scanning,or a photographing instruction is output to the CPU 190 from theoperating unit 180, the control unit 194 performs contrast scanning soas to focus on the edge of the subject image. In addition, thedistribution detection unit 193 calculates an Ed at a specific period.Further, the distribution detection unit 193 increases the rate ofcalculating the Ed if it is determined that a motion per unit time ofthe subject image is fast based on a motion vector of the subject imageincluded in the through-the-lens image. In addition, the distributiondetection unit 193 may increase the rate of calculating the Ed if it isdetermined that a relative difference between the evaluation value Edcalculated previously and the evaluation value Ed calculated this timeis large.

First, it is assumed that the control unit 194 performs contrastscanning at the number f1 to f2 of frames and enables the AF lens 112 toperform scanning driving. Here, the distribution detection unit 193increases a rate of calculating the Ed if it is determined that a motionper unit time of the subject image is fast during the scanning driving.

In addition, the control unit 194 drives the AF lens 112 to the focusposition at the number f2 to f3 of frames so as to focus on the commonedge of the subject image. Thereby, the common edge of the subject imageis in a focalized state at the number f3 to f4 of frames. Here, in acase where the subject distance varies, the focus position at the numberf3 to f4 of frames may be different from the focus position (initial).

Next, it is assumed that the Ed exceeds a predefined threshold value atthe number f4 of frames due to the variation in the subject distance.Thereby, the control unit 194 restricts a movement direction and amovement amount of the AF lens 112 based on the defocus-driving pulsetable.

For example, it is assumed that the lens position (focus position)output from the barrel control unit 118 is a value “1”, the evaluationvalue (defocus amount) detected by the distribution detection unit 193is a value “1”, and the direction index detected by the distributiondetection unit 193 indicates a front focus state.

In this case, the control unit 194 refers to whether or not “the numberof steps in front focus” corresponding to the lens position “1” and theevaluation value “1” is registered in the defocus-driving pulse table(refer to FIG. 24). In addition, in the example shown in FIG. 24, “thenumber of steps in front focus” corresponding to the lens position “1”and the evaluation value “1” is registered, and, therefore, the controlunit 194 acquires the number “23” of steps in front focus from thedefocus-driving pulse table.

Thereby, the control unit 194 predicts that there is a focus position ata position where the AF lens 112 is moved in a close direction from thepresent lens position by a movement amount corresponding to the number“23” of steps of driving pulses. In addition, the control unit 194outputs a photoelectric conversion surface to the barrel control unit118. Further, the control unit 194 drives the AF lens 112 in the closedirection by a movement amount corresponding to the number “23” of stepsof driving pulses. Thereby, it is assumed that the control unit 194enables the edge to be focused at the number f5 of frames.

In addition, it is assumed that the control unit 194 repeatedly performsthe same operation as at the number f3 to f5 of frames at “the number f5to f7 of frames” and “the number of f7 to f9 of frames”. Further, it isassumed that a user operates the oscillating weight 160 and thereby afocus instruction or a photographing instruction is input to the CPU 190at the number f10 of frames.

FIG. 28 is an enlarged view of the number f10 to first substrate 12 offrame part in the example of the focus drive shown in FIG. 27. Here, thelongitudinal axis indicates a lens position. In addition, the transverseaxis indicates the number of frames. The control unit 194 sets aprediction range of a focus position with respect to the focus positionbased on the defocus-driving pulse table (refer to FIG. 24) at thenumber f10 of frames. In addition, the control unit 194 performscontrast scanning for the prediction range. Further, the control unit194 temporarily passes the focus position where a contrast value shows apeak, and calculates the evaluation value Ed through interpolation atthe number f11 of frames.

In addition, at the number f11 to f12, the control unit 194 returns theAF lens 112 to the focus position calculated through the interpolation(focus position driving).

As described above, the focus adjusting device 191 includes the edgedetection unit 192 which detects an edge of a subject image for eachcolor component forming an image including the subject image which isincident from the lens barrel 111 having the AF lens 112 for performingfocus adjustment. In addition, the focus adjusting device 191 furtherincludes the distribution detection unit 193 which detects distributionsof a focalized state and an unfocused state in an image based on theedge detected for each color component by the edge detection unit 192.Further, the focus adjusting device 191 further includes the controlunit 194 which creates the “defocus-driving pulse table” includinghistory of the distributions detected by the distribution detection unit193 and moves the AF lens 112 so as to focus on the subject image basedon the history.

Thereby, the focus adjusting device 191 enables a subject image whichhas never been focused to be more rapidly focused than in a case wherethe defocus-driving pulse table is not referred to.

Third Embodiment

The third embodiment of the present invention will be described indetail with reference to the drawings. The third embodiment is differentfrom the first and second embodiments in that the distribution detectionunit 193 calculates a defocus amount by using difference data(auto-correlation) of pixel values due to pixel shift as an evaluationvalue. Hereinafter, only differences from the first and secondembodiments will be described.

FIG. 29 is a flowchart illustrating an operation of the focus adjustingdevice 191 having a moving image capturing mode. The CPU 190 sets theimaging apparatus 100 in a moving image capturing mode (step Se1). Here,the CPU 190 may set the imaging apparatus 100 in a moving imagecapturing mode based on an operation input of a user received by theoperating unit 180. Thereby, the CPU 190 controls the image processingunit 140 so as to perform an image process for an image captured by theimaging unit 110 as a moving image.

The image processing unit 140 acquires 1/30 decimation data (step Se2).Here, the 1/30 decimation data is a through-the-lens image which isoutput from the imaging element 119 every 1/30 seconds, and is athrough-the-lens image of which a resolution is reduced by decimating aportion thereof. In addition, the CPU 190 controls the imaging unit 110so as to execute an automatic exposure (AE) process (step Se3).

The control unit 194 of the focus adjusting device 191 executes a firstfocus drive. A focusing method in the focus drive may be appropriatelyselected. For example, the focusing method may be a contrast AF method,or may be other focusing methods (phase difference detection method andthe like). In addition, the edge detection unit 192 selects a region ofwhich a contrast of a color component amount is high from thethrough-the-lens image as an evaluation area for evaluating anevaluation value (step Se4).

The control unit 194 performs contrast scanning so as to enable the AFlens 112 perform scanning driving (step Se5). In addition, the edgedetection unit 192 shifts the through-the-lens image in the horizontaldirection by one pixel in the evaluation area. Further, the edgedetection unit 192 calculates (collects) difference data of pixel values(for example, color component amounts) between the through-the-lensimage shifted by one pixel in the horizontal direction and the originalthrough-the-lens image for each color channel (step Se6).

The distribution detection unit 193 creates a histogram of thedifference data for 10% of the number of the difference data of the Gchannel and calculates an average value of the difference data (stepSe7). Further, the distribution detection unit 193 may create ahistogram of the difference data for all the number of the differencedata of the G channel.

FIG. 30A shows a histogram of difference data in a focalized state. FIG.30B shows a histogram of difference data in an unfocused state. Thetransverse axis of FIGS. 30A and 30B indicates difference data. Inaddition, the longitudinal axis indicates a frequency of the differencedata.

Since a defocus amount is large in an unfocused state, a difference(adjacent difference) between pixel values of adjacent pixels is smallerthan an adjacent difference in a focalized state. For this reason, thehistogram is shifted to a side where the difference data is small.Therefore, an average value “Ave” of the difference data in a focalizedstate shown in FIG. 13A and an average value “Ave1” in an unfocusedstate shown in FIG. 13B have a magnitude correlation of “Ave>Ave1”. Forthis reason, in a case where the average value becomes smaller than apredefined threshold value, the distribution detection unit 193determines that the through-the-lens image is in an unfocused state.

Returning to FIG. 29, the description of the operation of the focusadjusting device 191 is continued. The image processing unit 140acquires new 1/30 decimation data (step Se8). In addition, the edgedetection unit 192 shifts the through-the-lens image in the horizontaldirection by one pixel in the evaluation area. Further, the edgedetection unit 192 calculates (collects) difference data of pixel values(for example, color component amounts) between the through-the-lensimage shifted by one pixel in the horizontal direction and the originalthrough-the-lens image for each color channel (step Se6). Thedistribution detection unit 193 creates a histogram of the differencedata for 10% of the number of the difference data and calculates anaverage value of the difference data (step Se9). Further, thedistribution detection unit 193 may create a histogram of the differencedata for all the number of the difference data.

In a case where the average value of the difference data of the Gchannel becomes smaller than a predefined threshold value, thedistribution detection unit 193 determines that the through-the-lensimage is in an unfocused (defocus) state (step Se10). In addition, thedistribution detection unit 193 determines whether the edge of thesubject image is in a front focus state or in a back focus state basedon variations in the average value of the difference data of the Rchannel and the average value of the difference data of the B channel(step Se11).

For example, if a variation in the average value of the difference dataof the R channel is larger than a variation in the average value of thedifference data of the B channel, the distribution detection unit 193may determine that the edge of the subject image is in a front focusstate.

In addition, the control unit 194 restricts scanning region and performscontrast scanning (step Se12).

As described above, the focus adjusting device 191 shifts the image byone pixel in the horizontal direction, calculates difference data withthe original image, and thereby restricts scanning region and performscontrast scanning Thereby, the control unit 194 finishes the focus drivein a short time with a light process load as compared with a case ofperforming normal contrast scanning Therefore, the focus adjustingdevice 191 enables a subject image which has never been focused to berapidly focused. As above, although the embodiments of the presentinvention have been described in detail with reference to the drawings,a detailed configuration is not limited to the embodiments. Appropriatemodifications are possible within the scope without departing from thespirit of the present invention.

For example, in a case of performing contrast scanning for a subjectimage having spot light (dot light source), there are a plurality ofpeaks of a contrast value. For this reason, there are many cases where afocus position cannot be restricted. Therefore, the focus adjustingdevice 191 may improve a performance of contrast AF of the spot lightsubject based on the above-described evaluation values.

FIG. 31 is a flowchart illustrating an operation of the focus adjustingdevice 191 of determining spot light. The image processing unit 140acquires a through-the-lens image (decimated VGA resolution) from theimaging unit 110 (step Sf1). Here, the period where the image processingunit 140 acquires a through-the-lens image may be a period (intervalrate) shorter than a normal period. In addition, the CPU 190 controlsthe imaging unit 110 so as to execute a debayer (color interpolation)process. Further, the image processing unit 140 executes image processesfor the acquired through-the-lens image. For example, the imageprocessing unit 140 executes a white balance adjustment process and anoise reduction process for the acquired through-the-lens image (stepSf2).

The distribution detection unit 193 creates a histogram of a colorcomponent amount and determines whether or not a clip channel exists inthe histogram (step Sf3). Here, the clip channel is a channel where acolor component amount is saturated. In addition, the distributiondetection unit 193 may create a histogram of a luminance value.

If there is the clip channel, the distribution detection unit 193determines that there is spot light in the vicinity of a pixel where theclip (saturation) occurs and searches for an effective edge in thevicinity thereof (step Sf4). In addition, in order to search for theeffective edge using a high resolution image (non-decimated image), theimage processing unit 140 acquires the high resolution image from theimaging unit 110 (multi-scale window). The distribution detection unit193 searches for the effective edge in the vicinity of the pixel wherethe clip (saturation) occurs in the high resolution image (step Sf5).

The distribution detection unit 193 calculates an LSF based oninformation of the edge and compares the LSF for each color channel asdescribed above. In addition, the distribution detection unit 193calculates a direction index and a defocus amount as described above(step Sf6). Further, the control unit 194 drives the AF lens 112 basedon the direction index and the defocus amount (step Sf7).

As such, the focus adjusting device 191 searches for an effective edgein the vicinity of a pixel where clip (saturation) occurs. As a result,it is possible to improve a performance of contrast AF of a spot lightsubject. In addition, the imaging apparatus 100 may change imagingconditions (for example, a diaphragm value, an exposure value, and thelike) based on a result of the focus adjusting device 191 detecting spotlight (scene detection).

In addition, for example, the imaging apparatus 100 switches an imagingmode into a “macro (close-up) imaging mode” based on the direction indexand the defocus amount calculated by the focus adjusting device 191.FIG. 32 is a diagram illustrating an operation of determining conversioninto a macro imaging mode. It is assumed that in a case where a positionof the AF lens 112 is at a closest location, the distribution detectionunit 193 determines that the subject image is in a front focus state. Inthis case, the CPU 190 of the imaging apparatus 100 switches an imagingmode into a “macro (close-up) imaging mode” based on the direction index(front focus state) and the defocus amount calculated by the focusadjusting device 191. Thereby, the control unit 194 restricts a scanningrange to a “macro (close-up) close region” which is located further on aclose side than the close side in the normal contrast scanning andperforms contrast scanning

As such, the imaging apparatus 100 can switch an imaging mode into a“macro (close-up) imaging mode” without depending on an operation inputof a user based on the direction index (front focus state) and thedefocus amount calculated by the focus adjusting device 191.

In addition, for example, the color channel may be expressed in a colorspace expression form in addition to RGB. For example, the color channelmay be expressed in a color space expression form using a colordifference (YCbCr, YUV, or the like).

Further, for example, the distribution detection unit 193 may subdivideblocks based on a result of grouping a subject image with the samecolor, and label the subject image.

Furthermore, for example, the distribution detection unit 193 may changea resolution of an image if an effective edge is intended to be desired.In addition, the distribution detection unit 193 may execute asuper-resolution process.

For example, each profile, each parameter, and each threshold valuedescribed above may be changed depending on a zoom position, an imageheight, and a focus range position.

In addition, for example, in a case where a variation in an evaluationvalue is determined as being small based on history of the evaluationvalue calculated at a specific period, the distribution detection unit193 may regard a motion of a subject as being slow and lengthen theperiod for calculating the evaluation value. In this way, thedistribution detection unit 193 can decrease a calculation load ofcalculating the evaluation value.

In addition, for example, an SFR (Spatial Frequency Response) may beused instead of the MTF.

(As to Method of Selecting Edge for Focus Adjustment from a Plurality ofEdges)

In a case where there are a plurality of edges in an image, the focusadjusting device selects predefined N (where N is an integer of 1 ormore) high-ranking edges from a plurality of edges according to apriority. In addition, the focus adjusting device 191 performs focusadjustment based on the selected edges (for example, refer to step S11of FIG. 19). As such, in a case where there is a plurality of edges inan image, a method in which the focus adjusting device selects an edgefor focus adjustment will be described below.

The distribution detection unit 193 (refer to FIG. 1) selectshigh-ranking edges in descending order of power (color component amount)from edges detected for each color component by the edge detection unit192. In addition, the distribution detection unit 193 selectshigh-ranking edges in descending order of contrast of the colorcomponent from edges detected for each color component by the edgedetection unit 192.

Here, in a case where edges of which contrasts of the color componentare the same and signal to noise ratios (S/N ratio) are different aremixed, the distribution detection unit 193 selects high-ranking edges indescending order of signal to noise ratio of the color component. Inaddition, in a case where edges of which the signal to noise ratios arethe same and contrasts of the color component are different are mixed,the distribution detection unit 193 selects high-ranking edges indescending order of signal to noise ratios of the color component.

Furthermore, in a case where signal to noise ratios of the color channelof edges are different, the distribution detection unit 193 selects atleast one of an edge having a relatively low signal to noise and arelatively high contrast, and an edge having a relatively high signal tonoise ratio and a relatively low contrast.

FIGS. 33A and 33B are diagrams illustrating a priority of an edgeselected in a case where edges having different signal to noise ratiosof the color component are mixed. The longitudinal axis indicates acolor component amount. In addition, the transverse axis indicates aposition (refer to FIG. 7) at the axis D. FIG. 33A shows an edge havinga relatively high signal to noise ratio and a relatively low contrast.

On the other hand, FIG. 33B shows an edge having a relatively low signalto noise ratio and a relatively high contrast. In a case where thesignal to noise ratios of the color channel of edges are not the same,at least one of the edges is selected by the distribution detection unit193.

The distribution detection unit 193 (refer to FIG. 1) selects an edgeformed by the color channels of black and white. In addition, in a casewhere there is no edge formed by the color channels of white and black,the distribution detection unit 193 selects an edge which includes twoor more primary colors and has a color component varying with the samephase from edges for each color component detected by the edge detectionunit 192. If two primary colors are included in an edge, thedistribution detection unit 193 selects an edge including the greencomponent (G channel). In addition, if the green component is notincluded in an edge, the distribution detection unit 193 selects an edgeincluding the blue component (B channel).

FIGS. 34A, 34B and 34C are diagrams illustrating a priority of an edgeselected when including two primary colors. The longitudinal axisindicates a color component amount. In addition, the transverse axisindicates a position (refer to FIG. 7) at the axis D. FIG. 34A shows anedge formed by the R channel and the G channel which vary with the samephase. FIG. 34B shows an edge formed by the R channel and the G channelvarying with the same phase, wherein a contrast of the R channel islower than the contrast of the R channel shown in FIG. 34A. In addition,FIG. 34C shows an edge formed by the R channel and the B channel varyingwith the same phase.

Among these edges, an edge having the highest priority is the edge,shown in FIG. 34A, which includes the green component (G channel) andhas a high contrast. Among these edges, an edge having the lowestpriority is the edge, shown in FIG. 34C, which does not include thegreen component. The edge including the green component is detected bythe imaging element 119 (refer to FIG. 1) with high accuracy.

The distribution detection unit 193 (refer to FIG. 1) selects an edgehaving a flat color component at a predefined width or more from theedges for color component detected by the edge detection unit 192. FIGS.35A and 35B are diagrams illustrating a variation in a color componentamount corresponding to a position crossing a single edge having a flatcolor component and a variation in a color component amountcorresponding to positions crossing a plurality of edges. Eachlongitudinal axis of FIGS. 35A and 35B indicates a color componentamount. In addition, the transverse axis of FIG. 35A indicates aposition crossing a single edge, that is, a position (refer to FIG. 7)at the axis D. Further, the transverse axis of FIG. 35B indicatespositions crossing a plurality of edges.

FIG. 35A shows a variation in a color component amount corresponding toa position crossing a single edge having a flat color component. In FIG.35A, the edge has the color component which is flat at predefined widthsW1 and W2 or more. Here, in relation to the width of the flat edge,instead of setting the widths W1 and W2, the widths W3 and W4 includinga range where the color component has a gradient may be set. On theother hand, FIG. 35B shows a variation in a color component amountcorresponding to positions crossing a plurality of edges. In FIG. 35B, acolor component amount of each edge is not flat. Therefore, an edgehaving the highest priority among these edges is the edge, shown in FIG.35A, having a flat color component at the predefined width or more.

The distribution detection unit 193 (refer to FIG. 1) selects an edgehaving a length or more defined according to the signal to noise ratioof the color component from edges which are detected for each colorcomponent by the edge detection unit 192. For example, the distributiondetection unit 193 selects a longer edge as the signal to noise ratio ofthe color component is lower.

Next, a procedure of selecting a high-ranking edge according to apriority will be described. FIG. 36 is a flowchart illustrating aprocedure of selecting a high-ranking edge according to the priority.The edge detection unit 192 detects a plurality of edges (step Sg1). Thedistribution detection unit 193 detects intensities (power, colorcomponent amount) of the edges detected by the edge detection unit 192(step Sg2). The edge detection unit 192 selects N edges which has a highpriority based on the detected intensities of the edges. For example,the edge detection unit 192 sets edges of which a color component amountis large as tracking targets (step Sg3).

The distribution detection unit 193 selects an edge having a highpriority based on the color components forming the selected edges (referto FIGS. 34A, 34B and 34C). For example, in a case where an edgeincludes two primary colors, the distribution detection unit 193 selectsan edge including the green component (G channel) (step Sg4).

The distribution detection unit 193 selects an edge having a highpriority based on a contrast of the selected edge (refer to FIG. 33).For example, the distribution detection unit 193 selects a high-rankingedge in descending order of a difference between color component amountsin adjacent pixels (adjacent difference) (step Sg5).

The distribution detection unit 193 selects an edge having a highpriority based on a signal to noise ratio of the color component of theselected edge, and the width of a range of being flat in a waveform(refer to FIGS. 35A and 35B) (step Sg6).

The distribution detection unit 193 selects an edge having a length ormore defined according to the signal to noise ratio of the colorcomponent. For example, the distribution detection unit 193 selects alonger edge as the signal to noise ratio of the color component is lower(step Sg7).

The distribution detection unit 193 registers an edge address which isidentification information of an edge. In other words, the distributiondetection unit 193 sets a high-ranking edge, which is finally selected,to a tracking target (step Sg8).

The control unit 194 determines whether the edge is in a front focusstate or in a back focus state, and a position of the AF lens 112, basedon the evaluation value of the edge. That is to say, the control unit194 moves the AF lens 112 so as to focus on the edge of the subjectimage based on the distributions of a focalized state and an unfocusedstate detected by the distribution detection unit 193. Here, the controlunit 194 restricts a movement direction of the AF lens 112 and performscontrast scanning (step Sg9).

In addition, the distribution detection unit 193 may select an edgehaving a high priority based on luminance of the edge instead of thecolor component of the edge.

In addition, a program for realizing the procedures described withreference to FIGS. 16, 19, 20, 23, 26, 29, 31 and 36 is recorded on acomputer readable recording medium, and a computer system is executed toread the program recorded on the recording medium, thereby performingthe execution processes. In addition, the “computer system” describedhere may include OS or hardware such as peripheral devices.

Here, the “computer system” includes home page providing circumstances(or display circumstances) if the WWW system is used. In addition, the“computer readable recording medium” refers to a flexible disk, amagneto-optical disc, a ROM, a writable nonvolatile memory such as aflash memory, a portable medium such as a CD-ROM, and a storage devicesuch as a hard disk embedded in the computer system.

Further, the “computer readable recording medium” includes a recordingmedium which holds a program for a specific time, such as a volatilememory (for example, DRAM (Dynamic Random Access Memory)) inside thecomputer system which is a server or a client in a case where theprogram is transmitted via a network such as the Internet or acommunication line such as a telephone line.

In addition, the program may be transmitted from the computer systemwhere the program is stored in a storage device or the like to othercomputer systems via a transmission medium or using a transmission wavein the transmission medium. Here, the “transmission medium” transmittingthe program refers to a medium having a feature of transmittinginformation such as a network (communication network) such as theInternet or a communication line (communication line) such as atelephone line.

In addition, the program may be a program for realizing a portion of theabove-described effects. In addition, the program may be a so-calleddifference file (difference program) which can realize theabove-described effects in combination with a program which has alreadyrecorded the above-described effects in the computer system.

REFERENCE SIGNS LIST

-   -   100 imaging apparatus    -   110 imaging unit    -   111 lens barrel    -   190 CPU    -   191 focus adjusting device    -   192 edge detection unit    -   193 distribution detection unit    -   194 control unit

1. A focus adjusting device comprising: an edge detection unit thatdetects an edge included in a subject from image data generated by animaging unit that images the subject; a distribution detection unit thatdetects distributions of a focalized state and an unfocused state basedon color component amounts of an edge detected by the edge detectionunit; and a control unit which moves a lens, that guides a light imageto the imaging unit, toward a direction so as to focus on an imagingsurface of the imaging unit based on the distributions detected by thedistribution detection unit.
 2. The focus adjusting device according toclaim 1, wherein the distribution detection unit detects distributionsof a focalized state and an unfocused state based on a ratio of colorcomponent amounts of an edge detected by the edge detection unit.
 3. Thefocus adjusting device according to claim 1, wherein the distributiondetection unit detects distributions of a focalized state and anunfocused state based on a difference of color component amounts of anedge detected by the edge detection unit.
 4. The focus adjusting deviceaccording to claim 1, wherein the distribution detection unit detects adirection for moving the lens.
 5. The focus adjusting device accordingto claim 1, wherein the distribution detection unit detects a movementamount for moving the lens.
 6. A focus adjusting device comprising: anedge detection unit that detects an edge included in a subject fromimage data generated by an imaging unit that images the subject; adistribution detection unit that detects distributions of a focalizedstate and an unfocused state based on a line spread function of an edgedetected by the edge detection unit; and a control unit which moves alens, that guides a light image to the imaging unit, toward a directionso as to focus on an imaging surface of the imaging unit based on thedistributions detected by the distribution detection unit.
 7. The focusadjusting device according to claim 6, wherein the distributiondetection unit detects distributions of a focalized state and anunfocused state based on a ratio of color component amounts of an edgedetected by the edge detection unit.
 8. The focus adjusting deviceaccording to claim 6, wherein the distribution detection unit detectsdistributions of a focalized state and an unfocused state based on adifference of color component amounts of an edge detected by the edgedetection unit.
 9. The focus adjusting device according to claim 6,wherein the distribution detection unit detects a direction for movingthe lens.
 10. The focus adjusting device according to claim 6, whereinthe distribution detection unit detects a movement amount for moving thelens.
 11. The focus adjusting device according to claim 6, wherein thedistribution detection unit detects distributions of a focalized stateand an unfocused state based on a standard deviation or a half maximumof a line spread function of an edge detected by the edge detectionunit.
 12. An imaging apparatus comprising: an imaging unit that images asubject; an edge detection unit that detects an edge included in thesubject from image data generated by the imaging unit; a distributiondetection unit that detects distributions of a focalized state and anunfocused state based on color component amounts of an edge detected bythe edge detection unit; and a control unit which moves a lens, thatguides a light image to the imaging unit, toward a direction so as tofocus on an imaging surface of the imaging unit based on thedistributions detected by the distribution detection unit.
 13. Theimaging apparatus according to claim 12, wherein the distributiondetection unit detects distributions of a focalized state and anunfocused state based on a ratio of color component amounts of an edgedetected by the edge detection unit.
 14. The imaging apparatus accordingto claim 12, wherein the distribution detection unit detectsdistributions of a focalized state and an unfocused state based on adifference of color component amounts of an edge detected by the edgedetection unit.
 15. The imaging apparatus according to claim 12, whereinthe distribution detection unit detects a direction for moving the lens.16. The imaging apparatus according to claim 12, wherein thedistribution detection unit detects a movement amount for moving thelens.